00010000:00010000:3.0:1785220580.954512:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a3d5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954525:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a3d5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954536:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a3d5c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954544:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a3d5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954555:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220580.954558:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000477bdee9/0xf62499ab927a3d1d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954563:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d892b6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.954566:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000477bdee9/0xf62499ab927a3d1d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954573:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000477bdee9/0xf62499ab927a3d1d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954582:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000477bdee9/0xf62499ab927a3d1d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x70ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954589:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000477bdee9/0xf62499ab927a3d1d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954608:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.954609:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.954611:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.954694:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220580.954698:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3d39 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954704:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.954708:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3d39 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954715:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3d39 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954726:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3d39 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x70f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954733:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3d39 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.954746:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.954747:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.954749:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220580.954791:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921f9c00/2, rc 0, transno 4295070279 00000100:00100000:3.0:1785220580.954804:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510511360:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.954817:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921fa680/2, rc 0, transno 4295070277 00000100:00100000:3.0:1785220580.954820:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510510464:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.954840:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8121a300/2, rc 0, transno 4295069684 00000100:00100000:3.0:1785220580.954848:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8121a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510509824:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.954855:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97840700/2, rc 0, transno 4295070278 00000100:00100000:3.0:1785220580.954858:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97840700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510507392:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.954865:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d977f0380/2, rc 0, transno 4295069681 00000100:00100000:3.0:1785220580.954868:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d977f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510507904:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.954874:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d977f2300/2, rc 0, transno 4295069682 00000100:00100000:3.0:1785220580.954877:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d977f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510508416:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.954884:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d977f1f80/2, rc 0, transno 4295069683 00000100:00100000:3.0:1785220580.954888:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d977f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510508032:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220580.957233:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957245:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957254:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957262:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957287:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220580.957523:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+906:3654:x1871938510510336:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18139us (18264us total) trans 4295070282 rc 0/0 00000100:00100000:3.0:1785220580.957532:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283734 00000100:00100000:3.0:1785220580.957554:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510512128 00000100:00100000:3.0:1785220580.957558:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510512128 00000100:00100000:3.0:1785220580.957581:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510512896 00000100:00100000:3.0:1785220580.957583:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510512896 00000100:00100000:3.0:1785220580.957597:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510512256 00000100:00100000:3.0:1785220580.957599:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510512256 00000100:00100000:3.0:1785220580.957611:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510513024 00000100:00100000:3.0:1785220580.957613:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510513024 00000100:00100000:3.0:1785220580.957625:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510513152 00000100:00100000:3.0:1785220580.957627:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510513152 00000100:00100000:3.0:1785220580.957642:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283744 00000100:00100000:3.0:1785220580.957649:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+831:3652:x1871938510512128:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220580.957670:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28916 count 1 00010000:00010000:3.0:1785220580.957686:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957702:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005aed4f2c/0xf62499ab92767510 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711aa expref: 622 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957709:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x70f4:0x0].0x0 since=49s v=(0 0) 00010000:00010000:3.0:1785220580.957713:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.957717:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957746:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005aed4f2c/0xf62499ab92767510 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711aa expref: 622 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957759:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005aed4f2c/0xf62499ab92767510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711aa expref: 622 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.957784:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510513280:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.957835:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:3.0:1785220580.957862:0:14170:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283745 00000100:00100000:3.0:1785220580.957869:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+908:3652:x1871938510512896:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220580.957880:0:14170:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:28977 count 1 00010000:00010000:3.0:1785220580.957892:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957903:0:14170:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007990dad5/0xf62499ab92772d40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75787 expref: 572 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957910:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7131:0x0].0x0 since=41s v=(0 0) 00010000:00010000:3.0:1785220580.957912:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7131:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.957916:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957932:0:14170:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007990dad5/0xf62499ab92772d40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75787 expref: 572 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.957947:0:14170:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007990dad5/0xf62499ab92772d40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75787 expref: 572 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.957958:0:14170:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510513408:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.957977:0:14170:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220580.957990:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283746 00000100:00100000:3.0:1785220580.957996:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+831:3652:x1871938510512256:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220580.958006:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28917 count 1 00010000:00010000:3.0:1785220580.958016:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958026:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000de5d85e/0xf62499ab92767923 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711f7 expref: 622 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958033:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x70f5:0x0].0x0 since=49s v=(0 0) 00010000:00010000:3.0:1785220580.958036:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.958040:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958054:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000de5d85e/0xf62499ab92767923 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711f7 expref: 622 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958063:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000de5d85e/0xf62499ab92767923 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711f7 expref: 622 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.958079:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510513536:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.958124:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00000100:00100000:3.0:1785220580.958392:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283747 00000100:00100000:3.0:1785220580.958403:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+908:3652:x1871938510513024:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220580.958416:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:28978 count 1 00010000:00010000:3.0:1785220580.958426:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958438:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cff1227d/0xf62499ab9277307a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7582f expref: 572 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958445:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7132:0x0].0x0 since=41s v=(0 0) 00010000:00010000:3.0:1785220580.958448:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7132:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.958452:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958468:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cff1227d/0xf62499ab9277307a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7582f expref: 572 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958477:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cff1227d/0xf62499ab9277307a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7582f expref: 572 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.958486:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98ade680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510513664:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.958510:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896900 going to sleep for 11 seconds 00010000:00010000:3.0:1785220580.958535:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220580.958541:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958549:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.958553:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958563:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958577:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958584:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a3d40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958602:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.958604:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.958605:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220580.958616:0:13869:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283748 00000100:00100000:3.0:1785220580.958623:0:13869:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+908:3655:x1871938510513152:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220580.958634:0:13869:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:28979 count 1 00010000:00010000:3.0:1785220580.958643:0:13869:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958659:0:13869:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000a897f25/0xf62499ab927732cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75898 expref: 572 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958666:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7133:0x0].0x0 since=41s v=(0 0) 00010000:00010000:3.0:1785220580.958668:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7133:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.958672:0:13869:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958686:0:13869:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000a897f25/0xf62499ab927732cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75898 expref: 572 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.958695:0:13869:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000a897f25/0xf62499ab927732cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75898 expref: 572 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.958705:0:13869:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98ade300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0000_UUID:13869:1871938510513792:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.958724:0:13869:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:3.0:1785220580.958981:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d96666a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510511744:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.959000:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a3d6a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x70f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.959786:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98c86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510513920:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220580.959812:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220580.961694:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510514688 00000100:00100000:3.0:1785220580.961700:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510514688 00000100:00100000:3.0:1785220580.961725:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510514560 00000100:00100000:3.0:1785220580.961729:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510514560 00000100:00100000:3.0:1785220580.961750:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283753 00000100:00100000:3.0:1785220580.961756:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a263b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+834:3654:x1871938510514560:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220580.961765:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28921 count 1 00010000:00010000:3.0:1785220580.961790:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a3dbe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.961805:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003fcc5ae2/0xf62499ab927683aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b713e8 expref: 622 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.961813:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x70f9:0x0].0x0 since=48s v=(0 0) 00010000:00010000:3.0:1785220580.961816:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.961821:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a3dbe lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.961850:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003fcc5ae2/0xf62499ab927683aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b713e8 expref: 622 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.961860:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003fcc5ae2/0xf62499ab927683aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b713e8 expref: 622 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.961869:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d96666a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510514944:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.963937:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99673 00000100:00100000:3.0:1785220580.963947:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97987b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+572:2406:x1871938519865088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220580.963958:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220580.963963:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006b4cc1f8/0xf62499ab927724cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x712e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75676 expref: 572 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.963972:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006b4cc1f8/0xf62499ab927724cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x712e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75676 expref: 572 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.963985:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006b4cc1f8/0xf62499ab927724cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x712e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75676 expref: 572 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.963996:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x712e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.963999:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x712e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.964004:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a3d63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x712e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.964012:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a3d63 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x712e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.964027:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220580.964395:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97987b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+571:2406:x1871938519865088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 450us (778us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220580.964402:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99673 00010000:00010000:3.0:1785220580.964421:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a3d6a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.964430:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a3d6a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.964442:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a3d6a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.964449:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a3d6a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.964471:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220580.964616:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d906f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+835:3653:x1871938510511232:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15573us (18077us total) trans 4295069685 rc 0/0 00000100:00100000:3.0:1785220580.964623:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283740 00000100:00100000:3.0:1785220580.965741:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220580.965754:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97843100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510515456:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220580.965801:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220580.975482:0:14252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dc639ce/0xf62499ab92772a0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75733 expref: 568 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.975495:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510512000:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.975513:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975551:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007990dad5/0xf62499ab92772d40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75787 expref: 567 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.975561:0:14170:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510513408:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.975573:0:14170:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975580:0:14170:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975587:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975595:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975602:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975703:0:14170:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220580.975800:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+913:3652:x1871938510512896:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17931us (19296us total) trans 4295070286 rc 0/0 00000100:00100000:3.0:1785220580.975812:0:14170:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283745 00010000:00010000:3.0:1785220580.975850:0:14208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000de5d85e/0xf62499ab92767923 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711f7 expref: 620 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.975861:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510513536:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.975873:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975887:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975894:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975902:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975910:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.975921:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220580.975993:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+836:3652:x1871938510512256:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17997us (19444us total) trans 4295069687 rc 0/0 00000100:00100000:3.0:1785220580.976000:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283746 00010000:00010000:3.0:1785220580.976028:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cff1227d/0xf62499ab9277307a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7582f expref: 566 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.976040:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98ade680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510513664:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.976052:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976059:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976066:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976073:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976080:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976115:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220580.976181:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+913:3652:x1871938510513024:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17780us (19601us total) trans 4295070287 rc 0/0 00000100:00100000:3.0:1785220580.976188:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283747 00010000:00010000:3.0:1785220580.976222:0:14200:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005aed4f2c/0xf62499ab92767510 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b711aa expref: 619 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.976233:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510513280:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.976245:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976253:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976260:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976269:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976275:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976283:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220580.976355:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+836:3652:x1871938510512128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18705us (19893us total) trans 4295069688 rc 0/0 00000100:00100000:3.0:1785220580.976361:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283744 00010000:00010000:3.0:1785220580.976378:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:3.0:1785220580.976383:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976391:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.976394:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976401:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976414:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976421:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a3d8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7131:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976437:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220580.976440:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976447:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.976449:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976456:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976465:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976472:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3d94 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976483:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220580.976486:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976493:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.976496:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976508:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976517:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976525:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a3d9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976539:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220580.976542:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976548:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.976551:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976558:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976567:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976574:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a3d86 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976604:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.976606:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.976608:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.976630:0:13869:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a897f25/0xf62499ab927732cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75898 expref: 565 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.976640:0:13869:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98ade300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0000_UUID:13869:1871938510513792:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.976652:0:13869:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976661:0:13869:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976667:0:13869:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976675:0:13869:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976682:0:13869:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976696:0:13869:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220580.976784:0:13869:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+913:3655:x1871938510513152:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18148us (20091us total) trans 4295070288 rc 0/0 00000100:00100000:3.0:1785220580.976791:0:13869:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283748 00010000:00010000:3.0:1785220580.976810:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220580.976814:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976821:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.976823:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976831:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976841:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976849:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a3da2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976863:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.976865:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.976867:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.976874:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976882:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976895:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976902:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.976914:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220580.976984:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98adce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+913:3655:x1871938510511872:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22911us (23666us total) trans 4295070289 rc 0/0 00000100:00100000:3.0:1785220580.976991:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283741 00010000:00010000:3.0:1785220580.977007:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220580.977011:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977018:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220580.977021:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977029:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977040:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977162:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3d71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7130:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977181:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.977183:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220580.977185:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220580.977213:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a705880/2, rc 0, transno 4295070285 00000100:00100000:3.0:1785220580.977226:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510514048:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.977288:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8af800/2, rc 0, transno 4295069687 00000100:00100000:3.0:1785220580.977307:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510512256:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.977319:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c87800/2, rc 0, transno 4295070287 00000100:00100000:3.0:1785220580.977325:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c87800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510513024:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.977358:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8af100/2, rc 0, transno 4295069688 00000100:00100000:3.0:1785220580.977363:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510512128:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.977372:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c85500/2, rc 0, transno 4295070286 00000100:00100000:3.0:1785220580.977378:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c85500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510512896:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220580.977412:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510516096:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220580.977447:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220580.977476:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510516096 00000100:00100000:3.0:1785220580.977481:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510516096 00000100:00100000:3.0:1785220580.977514:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283757 00000100:00100000:3.0:1785220580.977521:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90e8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+913:3652:x1871938510516096:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220580.977545:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:28985 count 1 00010000:00010000:3.0:1785220580.977565:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977584:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e473a601/0xf62499ab92774415 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75b93 expref: 564 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977592:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7139:0x0].0x0 since=40s v=(0 0) 00010000:00010000:3.0:1785220580.977596:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7139:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.977600:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977625:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e473a601/0xf62499ab92774415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75b93 expref: 564 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.977637:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e473a601/0xf62499ab92774415 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75b93 expref: 564 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.977650:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c928a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510516352:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.977703:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:3.0:1785220580.977718:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220580.977724:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510516224:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220580.977748:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220580.977824:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.979129:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.979159:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.979178:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519867008 00000100:00100000:3.0:1785220580.979182:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519867008 00010000:00010000:3.0:1785220580.979197:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003fcc5ae2/0xf62499ab927683aa lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b713e8 expref: 617 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.979217:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99686 00000100:00100000:3.0:1785220580.979223:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+617:2407:x1871938519867008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220580.979229:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220580.979234:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003fcc5ae2/0xf62499ab927683aa lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b713e8 expref: 617 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.979244:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003fcc5ae2/0xf62499ab927683aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b713e8 expref: 617 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.979257:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.979260:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.979265:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a3dbe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.979273:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a3dbe lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x70f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.979286:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220580.979301:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+617:2407:x1871938519867008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220580.979308:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99686 00000100:00100000:3.0:1785220580.979367:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.979832:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.980404:0:14206:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510516608 00000100:00100000:3.0:1785220580.980408:0:14206:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510516608 00000100:00100000:3.0:1785220580.981109:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283762 00000100:00100000:3.0:1785220580.981119:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97840000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+838:3654:x1871938510516608:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220580.981130:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28924 count 1 00010000:00010000:3.0:1785220580.981371:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.981383:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008d41143f/0xf62499ab9276b91f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b73fc4 expref: 613 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.981390:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x70fc:0x0].0x0 since=47s v=(0 0) 00010000:00010000:3.0:1785220580.981392:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.981396:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.981416:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008d41143f/0xf62499ab9276b91f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b73fc4 expref: 613 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.981425:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008d41143f/0xf62499ab9276b91f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b73fc4 expref: 613 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.981435:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a704000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510517376:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.981464:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220580.983364:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d96667800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510517760:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220580.983393:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220580.983424:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a262a00/2, rc 0, transno 4295069690 00000100:00100000:3.0:1785220580.983437:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a262a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510514688:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.983475:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c86d80/2, rc 0, transno 4295070284 00000100:00100000:3.0:1785220580.983485:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510513920:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.983519:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c85f80/2, rc 0, transno 4295070288 00000100:00100000:3.0:1785220580.983526:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c85f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510513152:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.983551:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c86680/2, rc 0, transno 4295070289 00000100:00100000:3.0:1785220580.983557:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c86680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510511872:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220580.983584:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510518400:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220580.983608:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220580.983624:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1785220580.983630:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510518272:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220580.983648:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220580.985405:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c928a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510516352:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.985427:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.985512:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.985528:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519867392 00000100:00100000:3.0:1785220580.985531:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519867392 00010000:00010000:3.0:1785220580.985549:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e473a601/0xf62499ab92774415 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75b93 expref: 563 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.985568:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99690 00000100:00100000:3.0:1785220580.985574:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c928a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+563:2406:x1871938519867392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220580.985584:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220580.985589:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e473a601/0xf62499ab92774415 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75b93 expref: 563 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985599:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e473a601/0xf62499ab92774415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75b93 expref: 563 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985617:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e473a601/0xf62499ab92774415 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75b93 expref: 563 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985630:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7139:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.985633:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7139:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.985638:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985649:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985668:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220580.985687:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c928a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+562:2406:x1871938519867392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220580.985694:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99690 00010000:00010000:3.0:1785220580.985716:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985724:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985734:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985742:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a3de1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7139:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.985756:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220580.987058:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90e8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+920:3652:x1871938510516096:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9535us (9609us total) trans 4295070292 rc 0/0 00000100:00100000:3.0:1785220580.987065:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283757 00000100:00100000:3.0:1785220580.987081:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510519808 00000100:00100000:3.0:1785220580.987101:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510519808 00000100:00100000:3.0:1785220580.987125:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283772 00000100:00100000:3.0:1785220580.987131:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a260e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+920:3654:x1871938510519808:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220580.988742:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:28991 count 1 00010000:00010000:3.0:1785220580.989171:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3e4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.989188:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cdd2c390/0xf62499ab9277528c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e79 expref: 561 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.989196:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x713f:0x0].0x0 since=40s v=(0 0) 00010000:00010000:3.0:1785220580.989200:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x713f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.989204:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3e4a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.989240:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cdd2c390/0xf62499ab9277528c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e79 expref: 561 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.989251:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cdd2c390/0xf62499ab9277528c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e79 expref: 561 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.989263:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93638700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510519936:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.989463:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:3.0:1785220580.989498:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510517376:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.989517:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.991037:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.991073:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.991125:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519868032 00000100:00100000:3.0:1785220580.991133:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519868032 00010000:00010000:3.0:1785220580.991167:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008d41143f/0xf62499ab9276b91f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b73fc4 expref: 613 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.991193:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519867904 00000100:00100000:3.0:1785220580.991196:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519867904 00010000:00010000:3.0:1785220580.991207:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004a0de880/0xf62499ab9276ba22 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x70fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7402d expref: 614 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.991228:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99694 00000100:00100000:3.0:1785220580.991235:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+614:2407:x1871938519868032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220580.991247:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220580.991251:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008d41143f/0xf62499ab9276b91f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b73fc4 expref: 614 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991262:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008d41143f/0xf62499ab9276b91f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b73fc4 expref: 614 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991281:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d41143f/0xf62499ab9276b91f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b73fc4 expref: 614 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991296:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.991300:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.991305:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991314:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991336:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220580.991369:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+613:2407:x1871938519868032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (339us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220580.991376:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99694 00000100:00100000:3.0:1785220580.991393:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99695 00000100:00100000:3.0:1785220580.991399:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94d54700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+612:2407:x1871938519867904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220580.991404:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220580.991408:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004a0de880/0xf62499ab9276ba22 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x70fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7402d expref: 612 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991417:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004a0de880/0xf62499ab9276ba22 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x70fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7402d expref: 612 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991428:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.991430:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x70fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.991434:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a3dfd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991441:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a3dfd lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x70fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991450:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220580.991464:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94d54700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+612:2407:x1871938519867904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 66us (391us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220580.991469:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99695 00010000:00010000:3.0:1785220580.991488:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991496:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991510:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991517:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a3e04 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.991532:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220580.991686:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97840000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+844:3654:x1871938510516608:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10568us (11573us total) trans 4295069695 rc 0/0 00000100:00100000:3.0:1785220580.991694:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283762 00000100:00100000:3.0:1785220580.991723:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.991848:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220580.991869:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220580.993498:0:14179:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021dc3946/0xf62499ab9276be35 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b740ff expref: 609 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.993509:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510518016:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220580.993520:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3e0b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.993528:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3e0b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.993535:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3e0b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.993543:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3e0b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x70ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.993549:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3e0b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x70ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.995651:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220580.995659:0:14179:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3e0b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x70ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.995851:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d96666a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+847:3652:x1871938510517504:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13132us (13192us total) trans 4295069699 rc 0/0 00000100:00100000:3.0:1785220580.995859:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283764 00000100:00100000:3.0:1785220580.995876:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510520192 00000100:00100000:3.0:1785220580.995880:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510520192 00000100:00100000:3.0:1785220580.995915:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283776 00000100:00100000:3.0:1785220580.995923:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+921:3654:x1871938510520192:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220580.995938:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:28992 count 1 00010000:00010000:3.0:1785220580.995951:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a3e66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.995964:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000052186c45/0xf62499ab9277555d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75ef0 expref: 559 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.995971:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7140:0x0].0x0 since=40s v=(0 0) 00010000:00010000:3.0:1785220580.995975:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7140:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220580.995979:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a3e66 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.996002:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000052186c45/0xf62499ab9277555d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75ef0 expref: 559 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220580.996013:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000052186c45/0xf62499ab9277555d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75ef0 expref: 559 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220580.996023:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c682d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510520960:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220580.996053:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00010000:00010000:3.0:1785220580.996070:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220580.996126:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8add7480/2, rc 0, transno 4295069699 00000100:00100000:3.0:1785220580.996138:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510517504:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.996151:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8add7100/2, rc 0, transno 4295069694 00000100:00100000:3.0:1785220580.996155:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510516864:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220580.996166:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8df80/2, rc 0, transno 4295070293 00000100:00100000:3.0:1785220580.996174:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510516224:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220580.996183:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8ea00/2, rc 0, transno 4295070292 00000100:00100000:3.0:1785220580.996186:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510516096:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.000936:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a705180/2, rc 0, transno 4295070296 00000100:00100000:3.0:1785220581.000957:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510518272:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.001186:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a707800/2, rc 0, transno 4295070295 00000100:00100000:3.0:1785220581.001191:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510518400:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.001230:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510522368:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.001264:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1785220581.001432:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1785220581.004718:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93638700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510519936:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.004738:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3e4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.005699:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519869184 00000100:00100000:3.0:1785220581.005707:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519869184 00010000:00010000:3.0:1785220581.005751:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000005f933d2/0xf62499ab92775055 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x713e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e3a expref: 559 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.005797:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99701 00000100:00100000:3.0:1785220581.005807:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+559:2405:x1871938519869184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.005822:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.005828:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000005f933d2/0xf62499ab92775055 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x713e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e3a expref: 559 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.005839:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000005f933d2/0xf62499ab92775055 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x713e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e3a expref: 559 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.005914:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005f933d2/0xf62499ab92775055 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x713e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e3a expref: 559 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.005937:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x713e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.005941:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x713e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.005947:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a3e43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x713e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.005957:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a3e43 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x713e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.005981:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.006023:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+557:2405:x1871938519869184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 217us (1044us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.006030:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99701 00010000:00010000:3.0:1785220581.006056:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3e4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.006066:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3e4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.006075:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3e4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.006104:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a3e4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.006125:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.006283:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a260e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+926:3654:x1871938510519808:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19150us (19615us total) trans 4295070297 rc 0/0 00000100:00100000:3.0:1785220581.006292:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283772 00000100:00100000:3.0:1785220581.008734:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.008754:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.008781:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.008802:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.008836:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.008855:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.009489:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519869824 00000100:00100000:3.0:1785220581.009493:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519869824 00010000:00010000:3.0:1785220581.009511:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b40a58a9/0xf62499ab9276d5e3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7109:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b745a5 expref: 607 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.011276:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99708 00000100:00100000:3.0:1785220581.011286:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c683b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+607:2407:x1871938519869824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.011295:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.011301:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b40a58a9/0xf62499ab9276d5e3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7109:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b745a5 expref: 607 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011312:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b40a58a9/0xf62499ab9276d5e3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7109:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b745a5 expref: 607 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011327:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b40a58a9/0xf62499ab9276d5e3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7109:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b745a5 expref: 607 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011343:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7109:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.011346:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7109:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.011350:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3e89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7109:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011358:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3e89 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7109:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011385:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.011407:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c683b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+606:2407:x1871938519869824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 123us (2549us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.011413:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99708 00000100:00100000:3.0:1785220581.011540:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.011560:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519868800 00000100:00100000:3.0:1785220581.011563:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519868800 00010000:00010000:3.0:1785220581.011580:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000072a047e1/0xf62499ab9276c59e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7102:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74272 expref: 606 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.011599:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99709 00000100:00100000:3.0:1785220581.011606:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+606:2407:x1871938519868800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.011613:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.011618:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000072a047e1/0xf62499ab9276c59e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7102:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74272 expref: 606 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011633:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000072a047e1/0xf62499ab9276c59e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7102:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74272 expref: 606 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011648:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000072a047e1/0xf62499ab9276c59e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7102:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74272 expref: 606 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011658:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7102:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.011661:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7102:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.011665:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a3e2e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7102:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011672:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a3e2e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7102:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.011688:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.011705:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+605:2407:x1871938519868800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (165us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.011711:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99709 00000100:00100000:3.0:1785220581.011750:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.011799:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.011973:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c682d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510520960:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.011991:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a3e66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.012058:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.012831:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.013446:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.016497:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a3e66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016506:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a3e66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016513:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a3e66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016519:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a3e66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7140:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016531:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.016613:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+931:3654:x1871938510520192:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20692us (21162us total) trans 4295070304 rc 0/0 00000100:00100000:3.0:1785220581.016619:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283776 00000100:00100000:3.0:1785220581.016635:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283790 00000100:00100000:3.0:1785220581.016642:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+852:3655:x1871938510524032:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.016651:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28939 count 1 00010000:00010000:3.0:1785220581.016662:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a3ec8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016676:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c25cb29b/0xf62499ab9276da90 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7469a expref: 602 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016683:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x710b:0x0].0x0 since=45s v=(0 0) 00010000:00010000:3.0:1785220581.016686:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x710b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.016690:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a3ec8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016724:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c25cb29b/0xf62499ab9276da90 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7469a expref: 602 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016733:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c25cb29b/0xf62499ab9276da90 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7469a expref: 602 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.016743:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510524928:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.016759:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.016907:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.016929:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519870592 00000100:00100000:3.0:1785220581.016933:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519870592 00010000:00010000:3.0:1785220581.016950:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000024aae7fa/0xf62499ab9276d1d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7107:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b744cc expref: 602 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.016970:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99715 00000100:00100000:3.0:1785220581.016976:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+602:2406:x1871938519870592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.016986:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.016989:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000024aae7fa/0xf62499ab9276d1d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7107:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b744cc expref: 602 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.016997:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000024aae7fa/0xf62499ab9276d1d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7107:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b744cc expref: 602 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.017010:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024aae7fa/0xf62499ab9276d1d0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7107:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b744cc expref: 602 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.017021:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7107:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.017024:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7107:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.017027:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a3e6d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7107:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.017034:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a3e6d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7107:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.017053:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.017067:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+601:2406:x1871938519870592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.017074:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99715 00000100:00100000:3.0:1785220581.018127:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.018214:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.018250:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.020278:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510523520:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.020304:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a3eac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.020338:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a99500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510523392:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.020350:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a3e9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7145:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.020520:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921f8380/2, rc 0, transno 4295069706 00000100:00100000:3.0:1785220581.020537:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510521088:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.020856:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519870848 00000100:00100000:3.0:1785220581.020860:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519870848 00010000:00010000:3.0:1785220581.020879:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008aee5a0a/0xf62499ab927763d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7145:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7610b expref: 550 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.020904:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99716 00000100:00100000:3.0:1785220581.020912:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+601:2406:x1871938519870720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.020926:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.020931:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000012c3d4ee/0xf62499ab9276d470 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7108:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74543 expref: 601 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.020942:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000012c3d4ee/0xf62499ab9276d470 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7108:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74543 expref: 601 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.020961:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7108:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.020966:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7108:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.020972:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a3e74 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7108:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.020983:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a3e74 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7108:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.020999:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.021045:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+601:2406:x1871938519870720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (2919us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.021053:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99716 00000100:00100000:3.0:1785220581.021071:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99718 00000100:00100000:3.0:1785220581.021077:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+549:2406:x1871938519870976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.021099:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.021105:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000216864dd/0xf62499ab9277696f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76215 expref: 549 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021115:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000216864dd/0xf62499ab9277696f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76215 expref: 549 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021129:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000216864dd/0xf62499ab9277696f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76215 expref: 549 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021140:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7147:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.021143:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7147:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.021147:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a3eac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021159:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a3eac lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021173:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.021191:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+548:2406:x1871938519870976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (2977us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.021197:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99718 00010000:00010000:3.0:1785220581.021214:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a3eac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021221:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a3eac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021228:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a3eac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021234:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a3eac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.021245:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.021323:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97a9b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+932:3655:x1871938510523136:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9676us (10073us total) trans 4295070306 rc 0/0 00000100:00100000:3.0:1785220581.021329:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283786 00000100:00100000:3.0:1785220581.023002:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99720 00000100:00100000:3.0:1785220581.023011:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+549:2407:x1871938519871232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.023019:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.023024:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008fbb4a90/0xf62499ab92776a95 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7148:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7627e expref: 548 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.023035:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008fbb4a90/0xf62499ab92776a95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7148:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7627e expref: 548 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.023048:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7148:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.023052:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7148:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.023056:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a3ea5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7148:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.023065:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a3ea5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7148:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.023077:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.024686:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+546:2407:x1871938519871232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1675us (2694us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.024695:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99720 00000100:00100000:3.0:1785220581.024732:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510524928:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.024746:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a3ec8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.025306:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283798 00000100:00100000:3.0:1785220581.025315:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d987c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+855:3652:x1871938510524416:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.025329:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28940 count 1 00010000:00010000:3.0:1785220581.025342:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025355:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c7453df1/0xf62499ab9276dc57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74718 expref: 598 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025363:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x710c:0x0].0x0 since=45s v=(0 0) 00010000:00010000:3.0:1785220581.025367:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x710c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.025372:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025390:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c7453df1/0xf62499ab9276dc57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74718 expref: 598 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025401:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c7453df1/0xf62499ab9276dc57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74718 expref: 598 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.025411:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510526976:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.025429:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.025444:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283800 00000100:00100000:3.0:1785220581.025451:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d8aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+936:3652:x1871938510526336:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.025462:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29007 count 1 00010000:00010000:3.0:1785220581.025473:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025489:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000037662b40/0xf62499ab92777d5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76579 expref: 545 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025496:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x714f:0x0].0x0 since=39s v=(0 0) 00010000:00010000:3.0:1785220581.025499:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x714f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.025502:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025519:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000037662b40/0xf62499ab92777d5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76579 expref: 545 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025529:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000037662b40/0xf62499ab92777d5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76579 expref: 545 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.025538:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510527488:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.025556:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.025605:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510527232 00000100:00100000:3.0:1785220581.025609:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510527232 00000100:00100000:3.0:1785220581.025632:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283801 00000100:00100000:3.0:1785220581.025638:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94939f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+856:3655:x1871938510527232:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.025649:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28944 count 1 00010000:00010000:3.0:1785220581.025660:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a3f15 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025673:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003105ef26/0xf62499ab9276e56b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b748fb expref: 598 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025681:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7110:0x0].0x0 since=44s v=(0 0) 00010000:00010000:3.0:1785220581.025684:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7110:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.025688:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a3f15 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025704:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003105ef26/0xf62499ab9276e56b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b748fb expref: 598 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.025715:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003105ef26/0xf62499ab9276e56b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b748fb expref: 598 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.025725:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510527616:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.025742:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.026056:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921fb100/2, rc 0, transno 4295069707 00000100:00100000:3.0:1785220581.026065:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510521216:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.026544:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99724 00000100:00100000:3.0:1785220581.026553:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+600:2406:x1871938519871744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.026561:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.026566:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006c1cba9d/0xf62499ab9276de5d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7478f expref: 600 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026574:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006c1cba9d/0xf62499ab9276de5d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7478f expref: 600 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026592:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006c1cba9d/0xf62499ab9276de5d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7478f expref: 600 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026602:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x710d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.026605:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x710d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.026609:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3ecf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026617:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3ecf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026633:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.026649:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+599:2406:x1871938519871744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.026655:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99724 00010000:00010000:3.0:1785220581.026707:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.026713:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3ecf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026720:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.026723:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3ecf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026731:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3ecf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026746:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3ecf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026753:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a3ecf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x710d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.026779:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.026781:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.026783:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.026858:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a705500/2, rc 0, transno 4295069709 00000100:00100000:3.0:1785220581.026866:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510524544:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.026901:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510528000:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.026945:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220581.027251:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97841500/2, rc 0, transno 4295070309 00000100:00100000:3.0:1785220581.027261:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510524160:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.027450:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510528128:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.027475:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.029962:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510525824:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.029976:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a3edd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x714d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.030265:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510527872 00000100:00100000:3.0:1785220581.030271:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510527872 00000100:00100000:3.0:1785220581.030314:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283806 00000100:00100000:3.0:1785220581.030322:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d96666a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+859:3652:x1871938510527872:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.030340:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28946 count 1 00010000:00010000:3.0:1785220581.030357:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a3f38 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7112:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030372:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c56fc134/0xf62499ab9276ea18 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7112:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74a1a expref: 598 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030381:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7112:0x0].0x0 since=44s v=(0 0) 00010000:00010000:3.0:1785220581.030385:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7112:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.030389:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a3f38 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7112:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030415:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c56fc134/0xf62499ab9276ea18 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7112:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74a1a expref: 598 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030427:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c56fc134/0xf62499ab9276ea18 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7112:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74a1a expref: 598 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.030439:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510528640:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.030494:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df180 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.030512:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.030519:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a3ec1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030527:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.030532:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a3ec1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030542:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a3ec1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030561:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a3ec1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030570:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a3ec1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.030592:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.030594:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.030596:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.031129:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a3ec8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.031148:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a3ec8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.031159:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a3ec8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.031167:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a3ec8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.031186:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.031352:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+860:3655:x1871938510524032:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14710us (15606us total) trans 4295069710 rc 0/0 00000100:00100000:3.0:1785220581.031359:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283790 00000100:00100000:3.0:1785220581.032820:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8ad180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8441:1871938510526464:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.032835:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a3eeb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x710f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.032867:0:14190:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c7453df1/0xf62499ab9276dc57 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74718 expref: 596 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.032880:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510526976:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.032895:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.032907:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.032915:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.032924:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.032971:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.032986:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.033148:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d987c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+861:3652:x1871938510524416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7825us (8506us total) trans 4295069711 rc 0/0 00000100:00100000:3.0:1785220581.033156:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283798 00010000:00010000:3.0:1785220581.033191:0:14208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037662b40/0xf62499ab92777d5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76579 expref: 546 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.033201:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510527488:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.033214:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033222:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033228:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033235:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033242:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033255:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.033348:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d8aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+939:3652:x1871938510526336:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7898us (8329us total) trans 4295070310 rc 0/0 00000100:00100000:3.0:1785220581.033355:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283800 00000100:00100000:3.0:1785220581.033383:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510527616:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.033396:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a3f15 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033412:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.033417:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033424:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.033428:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033437:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033451:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033458:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a3f00 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x710c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033493:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.033497:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033504:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.033508:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033516:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033527:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033535:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a3f0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.033546:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.033548:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.033550:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.039399:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a3f15 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.039408:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a3f15 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.039420:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a3f15 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.039426:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a3f15 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7110:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.039439:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.039544:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94939f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+861:3655:x1871938510527232:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13905us (14021us total) trans 4295069712 rc 0/0 00000100:00100000:3.0:1785220581.039551:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283801 00000100:00100000:3.0:1785220581.055183:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93761f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510529664:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.055198:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a3f54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7154:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.055684:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510532480:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.055698:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a3f9a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7119:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.058352:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519874688 00000100:00100000:3.0:1785220581.058357:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519874688 00010000:00010000:3.0:1785220581.058374:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb862331/0xf62499ab9276f299 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7117:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74c51 expref: 590 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.058400:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99743 00000100:00100000:3.0:1785220581.058407:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998baa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+590:2406:x1871938519874560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.058414:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.058418:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000075a09354/0xf62499ab9276ee24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7115:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74b39 expref: 590 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.058427:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000075a09354/0xf62499ab9276ee24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7115:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74b39 expref: 590 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.058439:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075a09354/0xf62499ab9276ee24 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7115:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74b39 expref: 590 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.058449:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7115:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.058451:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7115:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.058456:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a3f69 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7115:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.058464:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a3f69 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7115:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.058485:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.060999:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998baa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:2406:x1871938519874560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2587us (2857us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.061010:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99743 00000100:00100000:3.0:1785220581.061147:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.061215:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.061795:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.062815:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.062842:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.062935:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519874048 00000100:00100000:3.0:1785220581.062939:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519874048 00010000:00010000:3.0:1785220581.062958:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003b7c0fee/0xf62499ab92778a07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7153:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7677f expref: 536 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.062981:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99745 00000100:00100000:3.0:1785220581.062990:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+589:2406:x1871938519874944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.063004:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.063009:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f7ac0a02/0xf62499ab9276f8e3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74dcb expref: 589 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063020:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f7ac0a02/0xf62499ab9276f8e3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74dcb expref: 589 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063038:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f7ac0a02/0xf62499ab9276f8e3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74dcb expref: 589 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063050:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x711a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.063054:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x711a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.063060:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3f77 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063766:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3f77 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063802:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.063820:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+584:2406:x1871938519874944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 831us (2673us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.063827:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99745 00010000:00010000:3.0:1785220581.063869:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a3f9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7119:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063878:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a3f9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7119:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063886:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a3f9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7119:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063894:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a3f9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7119:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.063908:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.063990:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+869:3654:x1871938510530816:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15735us (17195us total) trans 4295069722 rc 0/0 00000100:00100000:3.0:1785220581.063997:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283820 00010000:00010000:3.0:1785220581.064015:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.064021:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a3f70 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7116:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064028:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.064032:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a3f70 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7116:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064041:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a3f70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7116:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064053:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a3f70 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7116:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064422:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a3f70 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7116:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064437:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.064439:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.064440:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.064469:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.064522:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.064643:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.064661:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519874816 00000100:00100000:3.0:1785220581.064664:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519874816 00010000:00010000:3.0:1785220581.064678:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd6ef091/0xf62499ab9276fae9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x711b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74e34 expref: 584 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.064702:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99750 00000100:00100000:3.0:1785220581.064709:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+534:2406:x1871938519874304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.064718:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.064722:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a1232f1b/0xf62499ab92778e6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7155:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76874 expref: 534 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064733:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a1232f1b/0xf62499ab92778e6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7155:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76874 expref: 534 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064748:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1232f1b/0xf62499ab92778e6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7155:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76874 expref: 534 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064760:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7155:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.064764:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7155:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.064781:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a3f5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7155:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064790:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a3f5b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7155:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064809:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.064828:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+533:2406:x1871938519874304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 119us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.064834:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99750 00000100:00100000:3.0:1785220581.064848:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99752 00000100:00100000:3.0:1785220581.064855:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+532:2406:x1871938519875072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.064862:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.064866:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c7637fc2/0xf62499ab92779242 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7157:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76954 expref: 532 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064881:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c7637fc2/0xf62499ab92779242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7157:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76954 expref: 532 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064895:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7637fc2/0xf62499ab92779242 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7157:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76954 expref: 532 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064904:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7157:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.064907:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7157:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.064911:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a3f93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7157:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064918:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a3f93 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7157:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.064935:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.064952:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+531:2406:x1871938519875072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.064958:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99752 00000100:00100000:3.0:1785220581.064999:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.065013:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519875328 00000100:00100000:3.0:1785220581.065016:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519875328 00010000:00010000:3.0:1785220581.065030:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000803f0c69/0xf62499ab9277983f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a49 expref: 531 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.065049:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99753 00000100:00100000:3.0:1785220581.065055:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+531:2406:x1871938519875328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.065341:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.065346:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000803f0c69/0xf62499ab9277983f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7159:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a49 expref: 531 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065356:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000803f0c69/0xf62499ab9277983f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7159:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a49 expref: 531 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065369:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000803f0c69/0xf62499ab9277983f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7159:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a49 expref: 531 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065379:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7159:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.065382:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7159:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.065386:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000db4e9c3a/0xf62499ab927a3fa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065395:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db4e9c3a/0xf62499ab927a3fa1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065411:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.065428:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+530:2406:x1871938519875328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 374us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.065435:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99753 00010000:00010000:3.0:1785220581.065452:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.065456:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3f85 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065467:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.065471:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3f85 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065479:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3f85 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x711b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065490:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3f85 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x711b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065497:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a3f85 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x711b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065515:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.065518:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3f77 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065524:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98dc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.065528:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3f77 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065535:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3f77 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065545:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3f77 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065624:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a3f77 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x711a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.065639:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.065641:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.065643:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.065680:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.065711:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.065760:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.065823:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.067136:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99757 00000100:00100000:3.0:1785220581.067144:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+531:2406:x1871938519875968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.067152:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.067157:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bdf03d84/0xf62499ab927790a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7156:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b768dd expref: 531 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.067166:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bdf03d84/0xf62499ab927790a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7156:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b768dd expref: 531 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.067179:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdf03d84/0xf62499ab927790a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7156:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b768dd expref: 531 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.067189:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7156:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.067192:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7156:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.067197:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a3f62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7156:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.067204:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a3f62 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7156:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.067222:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.067355:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+530:2406:x1871938519875968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 212us (1531us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.067363:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99757 00010000:00010000:3.0:1785220581.069493:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.069499:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a3fb6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.069506:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fea00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.069508:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a3fb6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.069516:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a3fb6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x711c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.069527:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a3fb6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x711c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.069534:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a3fb6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x711c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.069554:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.069556:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.069557:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.069588:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99271880/2, rc 0, transno 4295069726 00000100:00100000:3.0:1785220581.069592:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99271880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510533120:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.071544:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510534272:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.071591:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.074923:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519876096 00000100:00100000:3.0:1785220581.074927:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519876096 00010000:00010000:3.0:1785220581.074952:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000277dca56/0xf62499ab9277a011 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x715b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b37 expref: 528 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.074979:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99758 00000100:00100000:3.0:1785220581.074986:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+528:2406:x1871938519876096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.074996:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.075001:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000277dca56/0xf62499ab9277a011 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x715b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b37 expref: 528 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075013:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000277dca56/0xf62499ab9277a011 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x715b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b37 expref: 528 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075033:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000277dca56/0xf62499ab9277a011 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x715b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b37 expref: 528 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075054:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.075195:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.075202:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a3fbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075212:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a3fbd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x715b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075314:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.075340:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+527:2406:x1871938519876096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 353us (1079us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.075348:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99758 00000100:00100000:3.0:1785220581.075373:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510536192 00000100:00100000:3.0:1785220581.075376:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510536192 00000100:00100000:3.0:1785220581.075398:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510536320 00000100:00100000:3.0:1785220581.075400:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510536320 00000100:00100000:3.0:1785220581.075424:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510536448 00000100:00100000:3.0:1785220581.075427:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510536448 00000100:00100000:3.0:1785220581.075441:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510536576 00000100:00100000:3.0:1785220581.075444:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510536576 00000100:00100000:3.0:1785220581.075458:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510536704 00000100:00100000:3.0:1785220581.075461:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510536704 00000100:00100000:3.0:1785220581.075485:0:13869:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283836 00000100:00100000:3.0:1785220581.075494:0:13869:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d69a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+876:3653:x1871938510536192:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.075511:0:13869:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28962 count 1 00010000:00010000:3.0:1785220581.075529:0:13869:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075547:0:13869:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927707e6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75113 expref: 580 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075553:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7122:0x0].0x0 since=42s v=(0 0) 00010000:00010000:3.0:1785220581.075557:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7122:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.075561:0:13869:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075580:0:13869:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927707e6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75113 expref: 580 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075592:0:13869:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927707e6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75113 expref: 580 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.075603:0:13869:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510536832:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.075636:0:13869:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.075652:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283838 00000100:00100000:3.0:1785220581.075658:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d96666a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+955:3654:x1871938510536448:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.075670:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29026 count 1 00010000:00010000:3.0:1785220581.075682:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075693:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008c969cf5/0xf62499ab9277b446 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e8d expref: 526 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075704:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7162:0x0].0x0 since=37s v=(0 0) 00010000:00010000:3.0:1785220581.075707:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7162:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.075710:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075727:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008c969cf5/0xf62499ab9277b446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e8d expref: 526 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.075737:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008c969cf5/0xf62499ab9277b446 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e8d expref: 526 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.075746:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510537088:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.075778:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df180 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.078757:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c85c00/2, rc 0, transno 4295070327 00000100:00100000:3.0:1785220581.078870:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c85c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510533632:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.080824:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b032680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510537856:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.080852:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.083244:0:13869:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510536832:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.083259:0:13869:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.084116:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510537088:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.084131:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.084147:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519877248 00000100:00100000:3.0:1785220581.084150:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519877248 00010000:00010000:3.0:1785220581.084164:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927707e6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75113 expref: 579 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.084178:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519876864 00000100:00100000:3.0:1785220581.084180:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519876864 00010000:00010000:3.0:1785220581.084192:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006129a311/0xf62499ab92770611 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7121:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b750a3 expref: 581 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.084209:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519877376 00000100:00100000:3.0:1785220581.084211:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519877376 00010000:00010000:3.0:1785220581.084235:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bb335a7e/0xf62499ab9277b21d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7161:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e2b expref: 530 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.084253:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99762 00000100:00100000:3.0:1785220581.084258:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b00b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+530:2406:x1871938519876736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.084265:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.084269:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f213dd86/0xf62499ab9277a7c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x715e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76c5d expref: 530 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084278:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f213dd86/0xf62499ab9277a7c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x715e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76c5d expref: 530 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084295:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f213dd86/0xf62499ab9277a7c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x715e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76c5d expref: 529 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084305:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.084309:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.084312:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a3fee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084320:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a3fee lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x715e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084335:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.084374:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b00b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+528:2406:x1871938519876736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (1498us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.084381:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99762 00000100:00100000:3.0:1785220581.084399:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99763 00000100:00100000:3.0:1785220581.084405:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9af450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+527:2406:x1871938519876992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.084412:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.084418:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000052777d03/0xf62499ab9277aad0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d1a expref: 527 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084428:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000052777d03/0xf62499ab9277aad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d1a expref: 527 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084446:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052777d03/0xf62499ab9277aad0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d1a expref: 527 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084456:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.084459:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.084463:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084470:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.084487:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.087367:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9af450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+523:2406:x1871938519876992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2959us (4467us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.087376:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99763 00000100:00100000:3.0:1785220581.087433:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283845 00000100:00100000:3.0:1785220581.087441:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+958:3653:x1871938510538624:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.087451:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29029 count 1 00010000:00010000:3.0:1785220581.087462:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4049 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087473:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f831d90e/0xf62499ab9277bbee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76fe4 expref: 522 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087479:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7165:0x0].0x0 since=37s v=(0 0) 00010000:00010000:3.0:1785220581.087483:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7165:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.087487:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4049 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087504:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f831d90e/0xf62499ab9277bbee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76fe4 expref: 522 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087512:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f831d90e/0xf62499ab9277bbee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76fe4 expref: 522 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.087522:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510538880:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.087542:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.087562:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087571:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087580:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087589:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087603:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.087719:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+959:3654:x1871938510534656:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14241us (14312us total) trans 4295070330 rc 0/0 00000100:00100000:3.0:1785220581.087726:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283834 00010000:00010000:3.0:1785220581.087748:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.087753:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087760:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.087764:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087787:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087803:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 1/0,0 mode: --/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087810:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a3ffc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x715f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087827:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.087830:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087836:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.087839:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087851:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087862:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087869:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a400a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7122:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.087883:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.087885:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.087886:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.088038:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220581.088048:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510539136:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.088073:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.088358:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283847 00000100:00100000:3.0:1785220581.088366:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfa5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+881:3654:x1871938510538496:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.088381:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28968 count 1 00010000:00010000:3.0:1785220581.088394:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.088409:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000050889dd2/0xf62499ab927712ba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7537b expref: 575 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.088418:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7128:0x0].0x0 since=42s v=(0 0) 00010000:00010000:3.0:1785220581.088421:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7128:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.088426:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.088445:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000050889dd2/0xf62499ab927712ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7537b expref: 575 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.088456:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000050889dd2/0xf62499ab927712ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7537b expref: 575 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.088468:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510539392:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.088490:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df300 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.089797:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d87d46680/2, rc 0, transno 4295069732 00000100:00100000:3.0:1785220581.089814:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d87d46680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510536576:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.089918:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519877504 00000100:00100000:3.0:1785220581.089922:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519877504 00010000:00010000:3.0:1785220581.089941:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008c969cf5/0xf62499ab9277b446 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e8d expref: 523 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.089961:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519877760 00000100:00100000:3.0:1785220581.089963:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519877760 00010000:00010000:3.0:1785220581.089976:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009bba2faa/0xf62499ab92770b04 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7124:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b751c9 expref: 575 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.089996:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99769 00000100:00100000:3.0:1785220581.090003:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b00b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+523:2407:x1871938519877504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.090013:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.090017:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008c969cf5/0xf62499ab9277b446 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e8d expref: 523 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090026:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008c969cf5/0xf62499ab9277b446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e8d expref: 523 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090040:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c969cf5/0xf62499ab9277b446 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e8d expref: 523 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090049:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7162:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.090052:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7162:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.090057:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090064:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090082:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.090504:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b00b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+522:2407:x1871938519877504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 502us (665us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.090512:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99769 00000100:00100000:3.0:1785220581.090692:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519877888 00000100:00100000:3.0:1785220581.090696:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519877888 00010000:00010000:3.0:1785220581.090710:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007505dedc/0xf62499ab9277a37c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x715d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76bf4 expref: 523 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.090733:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519878016 00000100:00100000:3.0:1785220581.090735:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519878016 00010000:00010000:3.0:1785220581.090745:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b2922ae4/0xf62499ab9277a106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b76 expref: 524 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.090781:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99771 00000100:00100000:3.0:1785220581.090787:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+577:2406:x1871938519877120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.090792:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.090796:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000021779a07/0xf62499ab9276fff8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74f61 expref: 577 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090807:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000021779a07/0xf62499ab9276fff8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74f61 expref: 577 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090823:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021779a07/0xf62499ab9276fff8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74f61 expref: 577 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090835:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x711e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.090838:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x711e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.090843:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090851:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090875:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.090895:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+575:2406:x1871938519877120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.090901:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99771 00000100:00100000:3.0:1785220581.090915:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99772 00000100:00100000:3.0:1785220581.090921:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+524:2404:x1871938519878144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.090929:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.090934:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000717d7d59/0xf62499ab9277b5e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76ee1 expref: 524 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090944:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000717d7d59/0xf62499ab9277b5e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76ee1 expref: 524 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090956:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000717d7d59/0xf62499ab9277b5e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76ee1 expref: 524 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090966:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7163:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.090968:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7163:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.090972:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.090979:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091001:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.091017:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+523:2404:x1871938519878144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (604us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.091023:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99772 00000100:00100000:3.0:1785220581.091039:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99774 00000100:00100000:3.0:1785220581.091045:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+573:2404:x1871938519878400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.091051:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.091056:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008e06c923/0xf62499ab92770efb lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7126:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b752a2 expref: 573 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091066:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008e06c923/0xf62499ab92770efb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7126:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b752a2 expref: 573 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091077:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7126:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.091081:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7126:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.091145:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a403b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7126:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091155:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a403b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7126:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091168:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.091186:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+573:2404:x1871938519878400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 142us (747us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.091195:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99774 00000100:00100000:3.0:1785220581.091209:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99775 00000100:00100000:3.0:1785220581.091215:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+521:2404:x1871938519878016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.091234:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.091239:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b2922ae4/0xf62499ab9277a106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b76 expref: 521 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091248:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b2922ae4/0xf62499ab9277a106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b76 expref: 521 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091261:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2922ae4/0xf62499ab9277a106 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b76 expref: 521 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091270:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.091273:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x715c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.091277:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091285:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091301:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.091314:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+519:2404:x1871938519878016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.091320:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99775 00010000:00010000:3.0:1785220581.091339:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091347:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091354:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091360:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091370:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.091466:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d96666a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+961:3654:x1871938510536448:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15807us (16242us total) trans 4295070333 rc 0/0 00000100:00100000:3.0:1785220581.091474:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283838 00010000:00010000:3.0:1785220581.091495:0:14170:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091504:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091513:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091521:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091540:0:14170:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.091619:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94939f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+882:3655:x1871938510533504:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17947us (18008us total) trans 4295069733 rc 0/0 00000100:00100000:3.0:1785220581.091626:0:14170:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283835 00010000:00010000:3.0:1785220581.091646:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091654:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091662:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091669:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091681:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220581.091750:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95446450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+960:3654:x1871938510537472:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11768us (12153us total) trans 4295070334 rc 0/0 00000100:00100000:3.0:1785220581.091756:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283841 00000100:00100000:3.0:1785220581.091786:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510539904 00000100:00100000:3.0:1785220581.091791:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510539904 00000100:00100000:3.0:1785220581.091815:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510540032 00000100:00100000:3.0:1785220581.091818:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510540032 00000100:00100000:3.0:1785220581.091835:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510540160 00000100:00100000:3.0:1785220581.091837:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510540160 00000100:00100000:3.0:1785220581.091856:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283850 00000100:00100000:3.0:1785220581.091862:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+961:3653:x1871938510539904:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.091880:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29031 count 1 00010000:00010000:3.0:1785220581.091897:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091910:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000525cfda9/0xf62499ab9277c55d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77126 expref: 518 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091918:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7167:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.091921:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7167:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.091925:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091946:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000525cfda9/0xf62499ab9277c55d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77126 expref: 518 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.091958:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000525cfda9/0xf62499ab9277c55d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77126 expref: 518 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.091969:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510540288:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.092002:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.092014:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092022:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092031:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092038:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092046:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:3.0:1785220581.092667:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+962:3652:x1871938510534272:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20168us (21074us total) trans 4295070335 rc 0/0 00000100:00100000:3.0:1785220581.092673:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283829 00000100:00100000:3.0:1785220581.092688:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283853 00000100:00100000:3.0:1785220581.092695:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+884:3654:x1871938510539776:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.092705:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28970 count 1 00010000:00010000:3.0:1785220581.092716:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092727:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000041e9e531/0xf62499ab92771641 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7543f expref: 572 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092734:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x712a:0x0].0x0 since=42s v=(0 0) 00010000:00010000:3.0:1785220581.092737:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x712a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.092740:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092758:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000041e9e531/0xf62499ab92771641 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7543f expref: 572 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092779:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000041e9e531/0xf62499ab92771641 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7543f expref: 572 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.092790:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510540800:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.092817:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.092829:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:3.0:1785220581.092835:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a3fd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092842:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.092847:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a3fd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092856:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a3fd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x715d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092871:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a3fd2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x715d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092878:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a3fd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x715d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092899:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:3.0:1785220581.092904:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092910:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.092913:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092921:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092930:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092937:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4018 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092952:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:3.0:1785220581.092956:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092962:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.092965:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092973:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092982:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.092989:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4003 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x711e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093004:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.093007:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093013:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98cc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.093015:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093022:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093030:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093036:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a402d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7163:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093046:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.093049:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093054:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.093057:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093064:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093073:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093080:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a3fd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x715c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093120:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.093124:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009ae82ffe/0xf62499ab927a4026 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7124:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093131:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98ca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.093134:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009ae82ffe/0xf62499ab927a4026 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7124:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093141:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009ae82ffe/0xf62499ab927a4026 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7124:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093151:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009ae82ffe/0xf62499ab927a4026 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7124:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093158:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009ae82ffe/0xf62499ab927a4026 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7124:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093171:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.093173:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.093175:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.093367:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283854 00000100:00100000:3.0:1785220581.093375:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+963:3655:x1871938510540672:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.093389:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29033 count 1 00010000:00010000:3.0:1785220581.093402:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093413:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009d760658/0xf62499ab9277cf6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7721b expref: 518 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093421:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7169:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.093424:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7169:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.093428:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093447:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009d760658/0xf62499ab9277cf6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7721b expref: 518 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.093459:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009d760658/0xf62499ab9277cf6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7721b expref: 518 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.093471:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510541056:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.093523:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f240 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.093540:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.102960:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510539392:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.102976:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103008:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.103025:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510540800:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.103039:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103052:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.103057:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a405e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7166:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103063:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fee00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.103067:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a405e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7166:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103075:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a405e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7166:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103116:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a405e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7166:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103124:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a405e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7166:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103213:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.103215:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.103217:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.103309:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510540288:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.103322:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.103356:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4d880/2, rc 0, transno 4295070336 00000100:00100000:3.0:1785220581.103368:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510539136:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.103381:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b032680/2, rc 0, transno 4295069735 00000100:00100000:3.0:1785220581.103391:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b032680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510537856:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.103450:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98aded80/2, rc 0, transno 4295069734 00000100:00100000:3.0:1785220581.103457:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98aded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510536704:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.103490:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00bb80/2, rc 0, transno 4295070335 00000100:00100000:3.0:1785220581.103494:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510534272:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.103511:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99780 00000100:00100000:3.0:1785220581.103518:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+519:2407:x1871938519879040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.103525:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.103529:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000525cfda9/0xf62499ab9277c55d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77126 expref: 519 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103539:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000525cfda9/0xf62499ab9277c55d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77126 expref: 519 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103550:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000525cfda9/0xf62499ab9277c55d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77126 expref: 519 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103559:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7167:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.103562:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7167:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.103565:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103574:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.103588:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.104504:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+516:2407:x1871938519879040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 985us (1104us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.104513:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99780 00010000:00010000:3.0:1785220581.104537:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.104545:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.104553:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.104560:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a406c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7167:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.104572:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.104655:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+964:3653:x1871938510539904:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12793us (12995us total) trans 4295070338 rc 0/0 00000100:00100000:3.0:1785220581.104662:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283850 00010000:00010000:3.0:1785220581.104681:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.104693:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510543360:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.104723:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.104766:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220581.104785:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510543232:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.104807:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.107310:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107319:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107329:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107337:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107354:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.107517:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfa5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+887:3654:x1871938510538496:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19152us (19570us total) trans 4295069738 rc 0/0 00000100:00100000:3.0:1785220581.107527:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283847 00010000:00010000:3.0:1785220581.107545:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.107552:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107561:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.107565:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107576:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107590:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107598:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4057 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7128:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.107619:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.107621:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.107622:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.107641:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510538880:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.107658:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4049 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.107681:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510541056:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.107693:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.107745:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519879424 00000100:00100000:3.0:1785220581.107747:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519879424 00010000:00010000:3.0:1785220581.107765:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000041e9e531/0xf62499ab92771641 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7543f expref: 570 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.108378:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99785 00000100:00100000:3.0:1785220581.108385:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b00b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+516:2407:x1871938519879808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.108394:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.108399:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009d760658/0xf62499ab9277cf6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7721b expref: 516 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108409:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009d760658/0xf62499ab9277cf6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7721b expref: 516 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108422:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d760658/0xf62499ab9277cf6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7721b expref: 516 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108434:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7169:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.108437:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7169:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.108441:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108450:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108467:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.108484:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b00b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+514:2407:x1871938519879808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (842us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.108490:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99785 00000100:00100000:3.0:1785220581.108504:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99787 00000100:00100000:3.0:1785220581.108509:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+568:2406:x1871938519879424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.108518:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.108523:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000041e9e531/0xf62499ab92771641 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7543f expref: 568 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108532:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000041e9e531/0xf62499ab92771641 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7543f expref: 568 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108545:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000041e9e531/0xf62499ab92771641 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7543f expref: 568 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108555:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x712a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.108557:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x712a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.108561:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108569:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108589:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.108606:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+567:2406:x1871938519879424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1102us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.108612:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99787 00000100:00100000:3.0:1785220581.108655:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519879936 00000100:00100000:3.0:1785220581.108658:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519879936 00010000:00010000:3.0:1785220581.108673:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d0a50007/0xf62499ab9277d56a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x716a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7728b expref: 513 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.108695:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99788 00000100:00100000:3.0:1785220581.108701:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+513:2407:x1871938519879936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.108708:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.108712:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d0a50007/0xf62499ab9277d56a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x716a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7728b expref: 513 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108722:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d0a50007/0xf62499ab9277d56a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x716a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7728b expref: 513 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108735:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x716a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.108738:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x716a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.108742:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a408f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x716a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108749:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a408f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x716a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.108763:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.108793:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+513:2407:x1871938519879936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.108799:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99788 00010000:00010000:3.0:1785220581.111452:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111463:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111471:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111478:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4049 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111612:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.111699:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+968:3653:x1871938510538624:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24259us (24929us total) trans 4295070341 rc 0/0 00000100:00100000:3.0:1785220581.111707:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283845 00010000:00010000:3.0:1785220581.111730:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111738:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111753:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111759:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4088 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111790:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00000100:00100000:3.0:1785220581.111872:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+968:3655:x1871938510540672:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18499us (19150us total) trans 4295070342 rc 0/0 00000100:00100000:3.0:1785220581.111879:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283854 00010000:00010000:3.0:1785220581.111898:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111908:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111916:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111924:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4081 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x712a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.111943:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.112021:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90140380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+888:3654:x1871938510539776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19327us (19945us total) trans 4295069741 rc 0/0 00000100:00100000:3.0:1785220581.112031:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283853 00000100:00100000:3.0:1785220581.112058:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510544768:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.112081:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220581.112127:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90140700/2, rc 0, transno 4295070340 00000100:00100000:3.0:1785220581.112135:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90140700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510540928:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.113819:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90141880/2, rc 0, transno 4295070339 00000100:00100000:3.0:1785220581.113826:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510540032:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.113837:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a704e00/2, rc 0, transno 4295070341 00000100:00100000:3.0:1785220581.113840:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510538624:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.113952:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94938700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510545664:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.113986:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.114007:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.114012:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94939f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510545536:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.114031:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.115462:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.115481:0:13818:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510546560 00000100:00100000:3.0:1785220581.115485:0:13818:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510546560 00000100:00100000:3.0:1785220581.115509:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283869 00000100:00100000:3.0:1785220581.115515:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d935a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+891:3653:x1871938510546560:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.115526:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28979 count 1 00010000:00010000:3.0:1785220581.115540:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115552:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006edb160d/0xf62499ab9277317d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75867 expref: 565 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115559:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7133:0x0].0x0 since=41s v=(0 0) 00010000:00010000:3.0:1785220581.115562:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7133:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.115566:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115585:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006edb160d/0xf62499ab9277317d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75867 expref: 565 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115594:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006edb160d/0xf62499ab9277317d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75867 expref: 565 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.115604:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510546688:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.115628:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.115642:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283871 00000100:00100000:3.0:1785220581.115648:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+971:3655:x1871938510546048:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.115657:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29040 count 1 00010000:00010000:3.0:1785220581.115667:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115678:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f700123d/0xf62499ab9277e74c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b774fa expref: 511 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115685:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7170:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.115688:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7170:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.115691:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115706:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f700123d/0xf62499ab9277e74c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b774fa expref: 511 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.115715:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f700123d/0xf62499ab9277e74c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b774fa expref: 511 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.115725:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510546816:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.115746:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.116448:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.116486:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.117831:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a4096 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.117851:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a4096 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.117860:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a4096 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.117867:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a4096 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x712c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.117886:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.117980:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+892:3653:x1871938510541312:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21188us (21322us total) trans 4295069742 rc 0/0 00000100:00100000:3.0:1785220581.117986:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283856 00000100:00100000:3.0:1785220581.118613:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d69a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510547072:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.118648:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.125676:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a40c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x716c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.125686:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a40c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x716c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.125694:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a40c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x716c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.125702:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a40c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x716c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.125726:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.125910:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+974:3655:x1871938510543744:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19666us (19832us total) trans 4295070345 rc 0/0 00000100:00100000:3.0:1785220581.125918:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283862 00000100:00100000:3.0:1785220581.126319:0:14164:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510547840 00000100:00100000:3.0:1785220581.126325:0:14164:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510547840 00000100:00100000:3.0:1785220581.126396:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283875 00000100:00100000:3.0:1785220581.126405:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+893:3653:x1871938510547840:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.126424:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28981 count 1 00010000:00010000:3.0:1785220581.126443:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a411b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.126458:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d82e7f68/0xf62499ab92773638 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75947 expref: 563 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.126466:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7135:0x0].0x0 since=41s v=(0 0) 00010000:00010000:3.0:1785220581.126469:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7135:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.126473:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a411b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.126495:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d82e7f68/0xf62499ab92773638 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75947 expref: 563 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.126506:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d82e7f68/0xf62499ab92773638 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75947 expref: 563 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.126518:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510547968:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.126558:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.127544:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.128704:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510545792:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.128727:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.129049:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99795 00000100:00100000:3.0:1785220581.129059:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93608e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+564:2405:x1871938519880832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.129069:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.129074:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f9bb943e/0xf62499ab92772089 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x712d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b755d5 expref: 563 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.129112:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f9bb943e/0xf62499ab92772089 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x712d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b755d5 expref: 563 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.129128:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f9bb943e/0xf62499ab92772089 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x712d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b755d5 expref: 563 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.129139:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x712d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.129143:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x712d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.129146:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a40a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x712d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.129154:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a40a4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x712d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.129175:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.129208:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+562:2405:x1871938519880832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 151us (1175us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.129214:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99795 00000100:00100000:3.0:1785220581.132013:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510546688:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.132033:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.132066:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510546816:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.132078:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132185:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132194:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132205:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132213:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132231:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.132389:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+894:3653:x1871938510545664:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18325us (18401us total) trans 4295069746 rc 0/0 00000100:00100000:3.0:1785220581.132397:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283866 00000100:00100000:3.0:1785220581.132415:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510548352 00000100:00100000:3.0:1785220581.132419:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510548352 00000100:00100000:3.0:1785220581.132441:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510548480 00000100:00100000:3.0:1785220581.132444:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510548480 00000100:00100000:3.0:1785220581.132461:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283877 00000100:00100000:3.0:1785220581.132468:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+895:3655:x1871938510548352:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.132490:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28982 count 1 00010000:00010000:3.0:1785220581.132509:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4129 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132523:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006a4ec75f/0xf62499ab9277386f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b759e1 expref: 560 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132531:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7136:0x0].0x0 since=41s v=(0 0) 00010000:00010000:3.0:1785220581.132534:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7136:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.132539:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4129 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132565:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006a4ec75f/0xf62499ab9277386f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b759e1 expref: 560 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.132577:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006a4ec75f/0xf62499ab9277386f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b759e1 expref: 560 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.132590:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510548608:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.132642:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.133267:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99799 00000100:00100000:3.0:1785220581.133276:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+508:2406:x1871938519881856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.133285:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.133291:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f700123d/0xf62499ab9277e74c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b774fa expref: 508 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133302:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f700123d/0xf62499ab9277e74c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b774fa expref: 508 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133315:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f700123d/0xf62499ab9277e74c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b774fa expref: 508 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133324:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7170:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.133327:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7170:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.133331:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133338:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133358:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.133380:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+507:2406:x1871938519881856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.133387:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99799 00010000:00010000:3.0:1785220581.133408:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133416:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133424:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133432:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133446:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.133565:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90140000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+975:3655:x1871938510546048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17916us (17998us total) trans 4295070346 rc 0/0 00000100:00100000:3.0:1785220581.133572:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283871 00010000:00010000:3.0:1785220581.133592:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.133598:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133607:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.133611:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133621:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133635:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133643:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a40dc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7132:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133661:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.133664:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133670:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.133673:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133681:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133690:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133697:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a40ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133711:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.133713:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.133714:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.133957:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99801 00000100:00100000:3.0:1785220581.133965:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+562:2406:x1871938519881728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.133972:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.133978:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006edb160d/0xf62499ab9277317d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75867 expref: 562 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.133989:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006edb160d/0xf62499ab9277317d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75867 expref: 562 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134002:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006edb160d/0xf62499ab9277317d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75867 expref: 562 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134012:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7133:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.134015:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7133:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.134019:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134027:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134044:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.134055:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134063:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134071:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134079:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a40f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7133:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134117:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.134215:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d935a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+897:3653:x1871938510546560:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18700us (18749us total) trans 4295069747 rc 0/0 00000100:00100000:3.0:1785220581.134222:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283869 00000100:00100000:3.0:1785220581.134254:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+561:2406:x1871938519881728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 290us (647us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.134260:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99801 00000100:00100000:3.0:1785220581.134273:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99803 00000100:00100000:3.0:1785220581.134280:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+560:2405:x1871938519881216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.134309:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.134315:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aa990be8/0xf62499ab92772b6b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7131:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7576b expref: 560 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134325:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aa990be8/0xf62499ab92772b6b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7131:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7576b expref: 560 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134338:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa990be8/0xf62499ab92772b6b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7131:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7576b expref: 560 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134348:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7131:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.134351:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7131:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.134354:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a40e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134362:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a40e3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7131:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134385:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.134403:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+559:2405:x1871938519881216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 124us (695us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.134410:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99803 00010000:00010000:3.0:1785220581.134933:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.134939:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a40ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134947:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.134950:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a40ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134958:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a40ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134970:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a40ea lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134977:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a40ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.134993:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.134997:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a40f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135008:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.135010:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a40f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135017:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a40f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135027:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a40f8 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135033:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a40f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135046:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.135047:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.135049:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.135057:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519882240 00000100:00100000:3.0:1785220581.135060:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519882240 00010000:00010000:3.0:1785220581.135075:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000c7cbddc/0xf62499ab9277f212 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77690 expref: 506 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.135278:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99805 00000100:00100000:3.0:1785220581.135286:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce7a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+506:2405:x1871938519881344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.135295:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.135300:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006cc18b8e/0xf62499ab9277e4dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x716f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77498 expref: 506 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135310:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006cc18b8e/0xf62499ab9277e4dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x716f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77498 expref: 506 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135321:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006cc18b8e/0xf62499ab9277e4dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x716f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77498 expref: 506 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135332:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x716f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.135335:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x716f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.135339:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a40ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135347:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a40ce lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135365:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.135381:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce7a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+505:2405:x1871938519881344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (525us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.135387:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99805 00000100:00100000:3.0:1785220581.135420:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99807 00000100:00100000:3.0:1785220581.135427:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+504:2406:x1871938519882240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.135433:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.135438:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000c7cbddc/0xf62499ab9277f212 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7174:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77690 expref: 504 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135453:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000c7cbddc/0xf62499ab9277f212 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7174:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77690 expref: 504 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135465:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7174:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.135469:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7174:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.135474:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a410d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7174:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135482:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a410d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7174:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135494:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.135510:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+504:2406:x1871938519882240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.135515:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99807 00010000:00010000:3.0:1785220581.135532:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.135536:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a40d5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135543:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.135546:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a40d5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135553:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a40d5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135568:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a40d5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135575:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a40d5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x712f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.135590:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.135591:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.135593:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.135741:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.135924:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510547968:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.135937:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a411b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137601:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.137611:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4114 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7134:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137620:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.137623:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4114 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7134:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137632:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4114 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7134:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137653:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4114 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7134:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137659:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4114 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7134:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137679:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.137682:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a411b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137687:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.137690:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a411b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137697:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a411b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137705:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a411b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x7135:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137714:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.137717:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a40b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x716d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137724:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffe00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.137727:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a40b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x716d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137734:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a40b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x716d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137743:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a40b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x716d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.137751:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.137753:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.137755:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.137780:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510549376 00000100:00100000:3.0:1785220581.137787:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510549376 00000100:00100000:3.0:1785220581.137837:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510549504 00000100:00100000:3.0:1785220581.138130:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510549504 00000100:00100000:3.0:1785220581.138219:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510549248 00000100:00100000:3.0:1785220581.138223:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510549248 00000100:00100000:3.0:1785220581.138241:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510549888 00000100:00100000:3.0:1785220581.138244:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510549888 00000100:00100000:3.0:1785220581.138260:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510549760 00000100:00100000:3.0:1785220581.138262:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510549760 00000100:00100000:3.0:1785220581.138275:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510549120 00000100:00100000:3.0:1785220581.138277:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510549120 00000100:00100000:3.0:1785220581.138291:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510549632 00000100:00100000:3.0:1785220581.138293:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510549632 00000100:00100000:3.0:1785220581.138316:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283880 00000100:00100000:3.0:1785220581.138324:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+978:3652:x1871938510549376:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.138351:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29046 count 1 00010000:00010000:3.0:1785220581.138373:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138391:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ad8b4a2/0xf62499ab9277f5fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7777e expref: 503 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138399:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7176:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.138403:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7176:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.138407:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138436:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ad8b4a2/0xf62499ab9277f5fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7777e expref: 503 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138450:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ad8b4a2/0xf62499ab9277f5fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7777e expref: 503 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.138462:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510550016:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.138522:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.138542:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283881 00000100:00100000:3.0:1785220581.138551:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+899:3653:x1871938510549504:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.138563:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28987 count 1 00010000:00010000:3.0:1785220581.138577:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a4145 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138588:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007ece6b83/0xf62499ab927747b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75c96 expref: 555 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138598:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x713b:0x0].0x0 since=40s v=(0 0) 00010000:00010000:3.0:1785220581.138601:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x713b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.138605:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a4145 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138623:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007ece6b83/0xf62499ab927747b1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75c96 expref: 555 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138633:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007ece6b83/0xf62499ab927747b1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75c96 expref: 555 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.138645:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510550144:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.138667:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.138684:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283882 00000100:00100000:3.0:1785220581.138691:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90141f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+899:3655:x1871938510549248:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.138703:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28986 count 1 00010000:00010000:3.0:1785220581.138714:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138726:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d83cb1ad/0xf62499ab927744fc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75bee expref: 555 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138733:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x713a:0x0].0x0 since=40s v=(0 0) 00010000:00010000:3.0:1785220581.138736:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x713a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.138740:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138759:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d83cb1ad/0xf62499ab927744fc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75bee expref: 555 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.138796:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d83cb1ad/0xf62499ab927744fc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75bee expref: 555 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.138809:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510550272:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.138830:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.144422:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.145750:0:13836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006a4ec75f/0xf62499ab9277386f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b759e1 expref: 554 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.145764:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510548608:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.145796:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4129 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.145808:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4129 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.145816:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4129 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.145827:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4129 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.145836:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4129 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7136:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.145864:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.146017:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+902:3655:x1871938510548352:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13548us (14506us total) trans 4295069752 rc 0/0 00000100:00100000:3.0:1785220581.146024:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283877 00000100:00100000:3.0:1785220581.146286:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510550016:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.146302:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.146348:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510550144:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.146361:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a4145 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.146401:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510550272:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.146413:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.146496:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.146516:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.146534:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.146552:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.146571:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.146588:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.146610:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.147954:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a4145 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.147964:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a4145 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.147974:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a4145 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.147982:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a4145 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.148000:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.148199:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+902:3653:x1871938510549504:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9648us (11572us total) trans 4295069753 rc 0/0 00000100:00100000:3.0:1785220581.148208:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283881 00010000:00010000:3.0:1785220581.148234:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.148243:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.148251:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.148264:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.148280:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.150071:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+982:3652:x1871938510549376:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11747us (13700us total) trans 4295070354 rc 0/0 00000100:00100000:3.0:1785220581.150078:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283880 00010000:00010000:3.0:1785220581.150117:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150126:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150134:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150142:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150152:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/8 locks to regular blp list, 8 blwis in pool 00000100:00100000:3.0:1785220581.150251:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90141f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+904:3655:x1871938510549248:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11561us (13571us total) trans 4295069756 rc 0/0 00000100:00100000:3.0:1785220581.150258:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283882 00010000:00010000:3.0:1785220581.150273:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:3.0:1785220581.150279:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150287:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.150289:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150297:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150306:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150312:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a413e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150327:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:3.0:1785220581.150330:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4153 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150338:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.150341:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4153 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150349:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4153 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150358:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4153 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150367:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4153 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150379:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.150383:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a415a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150388:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.150392:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a415a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150400:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a415a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150409:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a415a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150416:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a415a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150427:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.150429:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150435:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.150438:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150444:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150453:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150460:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a414c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x713a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.150473:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.150475:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.150476:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.150670:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.150700:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.150725:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.150762:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.153342:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510553600 00000100:00100000:3.0:1785220581.153347:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510553600 00000100:00100000:3.0:1785220581.153376:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283900 00000100:00100000:3.0:1785220581.153383:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+907:3655:x1871938510553600:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.153394:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:28995 count 1 00010000:00010000:3.0:1785220581.153407:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a41ca lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7143:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.153417:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000da7667c1/0xf62499ab92775e78 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7143:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7602b expref: 549 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.153424:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7143:0x0].0x0 since=40s v=(0 0) 00010000:00010000:3.0:1785220581.153426:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7143:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.153429:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a41ca lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7143:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.153446:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000da7667c1/0xf62499ab92775e78 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7143:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7602b expref: 549 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.153454:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000da7667c1/0xf62499ab92775e78 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7143:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7602b expref: 549 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.153463:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510554368:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.153487:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.155738:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.155745:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a416f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.155752:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.155756:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a416f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.155764:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a416f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x713c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.155799:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a416f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x713c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.155805:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a416f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x713c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.155831:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.155833:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.155835:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.158354:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.164836:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfa5a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510553856:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.164863:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a41ae lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7142:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.164883:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519885056 00000100:00100000:3.0:1785220581.164886:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519885056 00010000:00010000:3.0:1785220581.164906:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000047038d39/0xf62499ab9277fddb lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b59 expref: 495 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.164934:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519885312 00000100:00100000:3.0:1785220581.164937:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519885312 00010000:00010000:3.0:1785220581.164949:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000071e1538d/0xf62499ab927751f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e48 expref: 548 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.164965:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519884544 00000100:00100000:3.0:1785220581.164968:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519884544 00010000:00010000:3.0:1785220581.164978:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000080d32e10/0xf62499ab9277faaf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779e6 expref: 496 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.164995:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99827 00000100:00100000:3.0:1785220581.165001:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+548:2406:x1871938519885312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.165007:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.165011:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000071e1538d/0xf62499ab927751f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e48 expref: 548 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165020:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000071e1538d/0xf62499ab927751f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e48 expref: 548 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165032:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000071e1538d/0xf62499ab927751f9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b75e48 expref: 548 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165042:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x713f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.165046:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x713f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.165049:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4199 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165057:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4199 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165074:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.165114:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4199 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165123:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4199 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165132:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4199 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165149:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4199 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x713f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165164:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.165404:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+910:3655:x1871938510552192:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15513us (15868us total) trans 4295069760 rc 0/0 00000100:00100000:3.0:1785220581.165412:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283893 00000100:00100000:3.0:1785220581.165444:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90140a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+547:2406:x1871938519885312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 443us (598us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.165451:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99827 00000100:00100000:3.0:1785220581.165479:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.165497:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.165523:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.165585:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.165602:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.165617:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.165647:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519886208 00000100:00100000:3.0:1785220581.165650:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519886208 00010000:00010000:3.0:1785220581.165663:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c83a08ac/0xf62499ab9277612d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7144:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76078 expref: 547 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.165820:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99832 00000100:00100000:3.0:1785220581.165828:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+497:2407:x1871938519885440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.165835:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.165840:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003a7cca5b/0xf62499ab9277fca0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ab8 expref: 497 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165849:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003a7cca5b/0xf62499ab9277fca0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ab8 expref: 497 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165861:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a7cca5b/0xf62499ab9277fca0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ab8 expref: 497 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165872:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x717e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.165875:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x717e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.165880:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a41c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165888:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a41c3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165905:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.165916:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a41c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165929:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a41c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165936:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a41c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165942:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a41c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.165954:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.166038:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+989:3655:x1871938510553088:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12994us (13291us total) trans 4295070360 rc 0/0 00000100:00100000:3.0:1785220581.166045:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283899 00000100:00100000:3.0:1785220581.166071:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+496:2407:x1871938519885440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 245us (573us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.166076:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99832 00000100:00100000:3.0:1785220581.166659:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99833 00010000:00010000:3.0:1785220581.166667:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.166671:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a41a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7141:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167493:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffe00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.167497:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a41a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7141:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167510:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a41a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7141:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167521:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a41a7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7141:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167529:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a41a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7141:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167555:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.167559:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a41e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167565:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fec00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.167568:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a41e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167576:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a41e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167587:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a41e6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167594:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a41e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167613:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.167616:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a41ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7142:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167623:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.167626:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a41ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7142:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167633:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a41ae lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7142:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167642:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a41ae lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7142:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167649:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a41ae lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7142:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167661:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.167663:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.167664:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.167739:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90700450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+491:2407:x1871938519885696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.167748:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.167755:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000036326f40/0xf62499ab9277fbb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a3a expref: 491 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167780:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000036326f40/0xf62499ab9277fbb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a3a expref: 491 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167792:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036326f40/0xf62499ab9277fbb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a3a expref: 491 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167801:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x717d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.167805:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x717d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.167809:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a41a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167816:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a41a0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167836:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.167848:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a41a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167857:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a41a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167866:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a41a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167874:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a41a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.167893:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.167973:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c928a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+989:3653:x1871938510552832:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17408us (17465us total) trans 4295070362 rc 0/0 00000100:00100000:3.0:1785220581.167979:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283894 00000100:00100000:3.0:1785220581.167992:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510556032 00000100:00100000:3.0:1785220581.167996:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510556032 00000100:00100000:3.0:1785220581.168022:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283907 00000100:00100000:3.0:1785220581.168028:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+989:3653:x1871938510556032:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.168047:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29060 count 1 00010000:00010000:3.0:1785220581.168064:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a41fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.168076:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f4a4fca7/0xf62499ab92780306 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77dd6 expref: 490 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.168083:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7184:0x0].0x0 since=35s v=(0 0) 00010000:00010000:3.0:1785220581.168114:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7184:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.168118:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a41fb lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.168152:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f4a4fca7/0xf62499ab92780306 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77dd6 expref: 490 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.168164:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f4a4fca7/0xf62499ab92780306 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77dd6 expref: 490 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.168181:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510556160:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.168799:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.174596:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93608a80/2, rc 0, transno 4295069767 00000100:00100000:3.0:1785220581.174611:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93608a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510555520:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.174853:0:14168:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510559360 00000100:00100000:3.0:1785220581.174860:0:14168:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510559360 00000100:00100000:3.0:1785220581.174898:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283920 00000100:00100000:3.0:1785220581.174906:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b00b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+917:3655:x1871938510559360:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.174925:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29005 count 1 00010000:00010000:3.0:1785220581.174943:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.174958:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000231e67d8/0xf62499ab927774e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76445 expref: 539 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.174966:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x714d:0x0].0x0 since=39s v=(0 0) 00010000:00010000:3.0:1785220581.174970:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x714d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.174974:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.174996:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000231e67d8/0xf62499ab927774e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76445 expref: 539 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.175008:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000231e67d8/0xf62499ab927774e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76445 expref: 539 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.175020:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8ad180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510559488:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.175067:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.180504:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.180526:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.180541:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.180554:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.180592:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.181282:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.181574:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99843 00000100:00100000:3.0:1785220581.181585:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+489:2406:x1871938519887744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.181593:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.181598:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b56c79c4/0xf62499ab927804bf lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e9a expref: 489 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181608:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b56c79c4/0xf62499ab927804bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e9a expref: 489 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181621:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7186:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.181624:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7186:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.181628:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a422c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181635:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a422c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181655:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.181671:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+489:2406:x1871938519887744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (1116us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.181677:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99843 00000100:00100000:3.0:1785220581.181690:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99845 00000100:00100000:3.0:1785220581.181695:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+488:2406:x1871938519888128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.181702:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.181706:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000009211230/0xf62499ab927806d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f73 expref: 488 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181715:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000009211230/0xf62499ab927806d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f73 expref: 488 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181726:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009211230/0xf62499ab927806d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f73 expref: 488 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181736:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7188:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.181739:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7188:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.181743:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a424f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181750:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a424f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.181783:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.181800:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+486:2406:x1871938519888128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.181806:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99845 00010000:00010000:3.0:1785220581.182343:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.182349:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a4217 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7146:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.182360:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.182364:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a4217 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7146:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.182372:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a4217 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7146:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.182386:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a4217 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7146:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.182392:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a4217 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7146:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.182411:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.182413:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.182415:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.183524:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.183530:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a424f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.183537:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.183540:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a424f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.183547:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a424f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.183558:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a424f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.183565:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a424f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.183580:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.183581:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.183583:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.184016:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9360a300/2, rc 0, transno 4295069770 00000100:00100000:3.0:1785220581.184021:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510556544:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.184123:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9360b800/2, rc 0, transno 4295070369 00000100:00100000:3.0:1785220581.184128:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510557696:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.184218:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.184222:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a4210 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7148:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184235:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.184238:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a4210 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7148:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184245:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a4210 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7148:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184257:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a4210 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7148:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184264:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a4210 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7148:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184280:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.184284:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a422c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184290:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.184293:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a422c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184299:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a422c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184308:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a422c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184315:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a422c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184328:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.184331:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4248 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184337:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.184340:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4248 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184347:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4248 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184357:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4248 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184364:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4248 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184377:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.184379:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.184381:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.184408:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ae300/2, rc 0, transno 4295070370 00000100:00100000:3.0:1785220581.184411:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510558080:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.184848:0:13874:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510560128 00000100:00100000:3.0:1785220581.184852:0:13874:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510560128 00000100:00100000:3.0:1785220581.184876:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283924 00000100:00100000:3.0:1785220581.184881:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+920:3655:x1871938510560256:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.184889:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29007 count 1 00010000:00010000:3.0:1785220581.184906:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184922:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000d33634/0xf62499ab92777b90 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7652c expref: 536 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184932:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x714f:0x0].0x0 since=39s v=(0 0) 00010000:00010000:3.0:1785220581.184936:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x714f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.184941:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184965:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000d33634/0xf62499ab92777b90 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7652c expref: 536 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.184979:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000d33634/0xf62499ab92777b90 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7652c expref: 536 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.184991:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a9b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510560512:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.186696:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.186785:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8ad180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510559488:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.186813:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189254:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189264:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189274:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189283:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189299:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.189453:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b00b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+922:3655:x1871938510559360:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14547us (14728us total) trans 4295069771 rc 0/0 00000100:00100000:3.0:1785220581.189461:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283920 00010000:00010000:3.0:1785220581.189482:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.189488:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189495:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.189498:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189514:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189525:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189532:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4256 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x714d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189553:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.189557:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4202 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189564:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.189567:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4202 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189575:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4202 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189582:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4202 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189589:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4202 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7147:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189607:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.189610:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a425d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189616:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.189618:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a425d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189626:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a425d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189635:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a425d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189642:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a425d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.189654:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.189656:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.189658:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.189664:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.190005:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510561664 00000100:00100000:3.0:1785220581.190009:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510561664 00000100:00100000:3.0:1785220581.190036:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510561792 00000100:00100000:3.0:1785220581.190039:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510561792 00000100:00100000:3.0:1785220581.190056:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283930 00000100:00100000:3.0:1785220581.190062:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9147ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+922:3652:x1871938510561792:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.190076:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29010 count 1 00010000:00010000:3.0:1785220581.190126:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.190145:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000070f37693/0xf62499ab92778426 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b766c2 expref: 534 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.190153:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7152:0x0].0x0 since=38s v=(0 0) 00010000:00010000:3.0:1785220581.190156:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7152:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.190159:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.190174:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000070f37693/0xf62499ab92778426 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b766c2 expref: 534 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.190183:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000070f37693/0xf62499ab92778426 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b766c2 expref: 534 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.190195:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0001_UUID:14232:1871938510562176:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.190238:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.191406:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.191440:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.191469:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.191510:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.191542:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.191565:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.192686:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99853 00000100:00100000:3.0:1785220581.192696:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97840000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+484:2404:x1871938519886848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.192704:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.192709:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000088188025/0xf62499ab9278038b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7185:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e31 expref: 484 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192719:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000088188025/0xf62499ab9278038b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7185:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e31 expref: 484 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192730:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088188025/0xf62499ab9278038b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7185:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e31 expref: 483 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192740:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7185:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.192742:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7185:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.192747:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4233 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192755:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4233 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192785:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.192802:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97840000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+481:2404:x1871938519886848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (1291us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.192809:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99853 00000100:00100000:3.0:1785220581.192822:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99855 00000100:00100000:3.0:1785220581.192829:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+535:2404:x1871938519887872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.192836:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.192904:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000054e9b4d2/0xf62499ab92776ca9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b762ee expref: 535 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192914:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000054e9b4d2/0xf62499ab92776ca9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b762ee expref: 535 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192926:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054e9b4d2/0xf62499ab92776ca9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b762ee expref: 535 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192936:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x714a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.192939:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x714a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.192942:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a4225 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192949:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a4225 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.192966:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.192984:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+534:2404:x1871938519887872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 156us (1418us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.192990:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99855 00010000:00010000:3.0:1785220581.193011:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.193017:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a423a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193029:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.193033:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a423a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193044:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a423a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x714c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193057:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a423a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x714c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193065:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a423a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x714c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193079:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.193083:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a41fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193118:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.193122:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a41fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193129:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a41fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193244:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a41fb lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193252:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a41fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193268:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.193272:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a4225 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193279:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98da00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.193282:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a4225 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193289:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a4225 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193299:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a4225 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193307:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a4225 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x714a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193320:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.193323:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4241 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193329:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.193333:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4241 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193341:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4241 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193352:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4241 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193360:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4241 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193373:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.193376:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a4209 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193381:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98cc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.193384:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a4209 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193391:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a4209 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193401:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a4209 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193408:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a4209 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7149:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193420:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.193421:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.193423:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.193478:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.193531:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a9b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510560512:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.193547:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.193631:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.193646:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519888768 00000100:00100000:3.0:1785220581.193650:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519888768 00010000:00010000:3.0:1785220581.193664:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009be97254/0xf62499ab92777983 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x714e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b764c3 expref: 533 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.193681:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99857 00000100:00100000:3.0:1785220581.193687:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+533:2405:x1871938519888768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.193694:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.193697:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009be97254/0xf62499ab92777983 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x714e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b764c3 expref: 533 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193707:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009be97254/0xf62499ab92777983 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x714e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b764c3 expref: 533 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193718:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x714e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.193722:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x714e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.193726:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4279 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193732:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4279 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x714e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.193743:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.193759:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+533:2405:x1871938519888768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 73us (128us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.193780:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99857 00000100:00100000:3.0:1785220581.193996:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510562432 00000100:00100000:3.0:1785220581.194000:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510562432 00000100:00100000:3.0:1785220581.194031:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510562688 00000100:00100000:3.0:1785220581.194034:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510562688 00000100:00100000:3.0:1785220581.194050:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510562560 00000100:00100000:3.0:1785220581.194052:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510562560 00000100:00100000:3.0:1785220581.194070:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283932 00000100:00100000:3.0:1785220581.194076:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1002:3654:x1871938510562432:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.194249:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29072 count 1 00010000:00010000:3.0:1785220581.194268:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a42aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194281:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f24b59da/0xf62499ab9278115a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783f6 expref: 478 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194289:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7190:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.194292:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7190:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.194296:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a42aa lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194318:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f24b59da/0xf62499ab9278115a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783f6 expref: 478 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194329:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f24b59da/0xf62499ab9278115a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783f6 expref: 478 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.194340:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97840e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510562816:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.194375:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.194390:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283933 00000100:00100000:3.0:1785220581.194398:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1002:3653:x1871938510562688:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.194411:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29073 count 1 00010000:00010000:3.0:1785220581.194420:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a42b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194432:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000016c96027/0xf62499ab927811fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78458 expref: 478 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194439:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7191:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.194442:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7191:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.194446:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a42b1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194462:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000016c96027/0xf62499ab927811fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78458 expref: 478 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.194472:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000016c96027/0xf62499ab927811fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78458 expref: 478 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.194482:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510562944:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.199427:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.200212:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200222:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200233:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200241:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200261:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.200419:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+927:3655:x1871938510560256:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15537us (15778us total) trans 4295069778 rc 0/0 00000100:00100000:3.0:1785220581.200427:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283924 00010000:00010000:3.0:1785220581.200448:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.200455:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200469:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.200472:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200481:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200492:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200500:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a426b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x714f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200518:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.200521:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.200522:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.200540:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0001_UUID:14232:1871938510562176:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.200557:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.200575:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519889152 00000100:00100000:3.0:1785220581.200579:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519889152 00010000:00010000:3.0:1785220581.200595:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001d8106d7/0xf62499ab92780d8d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78252 expref: 479 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.200627:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519889408 00000100:00100000:3.0:1785220581.200630:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519889408 00010000:00010000:3.0:1785220581.200642:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000070f37693/0xf62499ab92778426 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b766c2 expref: 529 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.200659:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519889280 00000100:00100000:3.0:1785220581.200662:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519889280 00010000:00010000:3.0:1785220581.200672:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003ebea4ba/0xf62499ab92780f23 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b782c9 expref: 480 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.200688:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519889536 00000100:00100000:3.0:1785220581.200690:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519889536 00010000:00010000:3.0:1785220581.200700:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ee9bd2d8/0xf62499ab92778140 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7151:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76660 expref: 530 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.200717:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99860 00000100:00100000:3.0:1785220581.200723:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+479:2407:x1871938519889152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.200732:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.200736:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001d8106d7/0xf62499ab92780d8d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78252 expref: 479 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200745:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001d8106d7/0xf62499ab92780d8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78252 expref: 479 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200757:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x718d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.200761:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x718d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.200765:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200784:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4280 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200797:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.200818:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+479:2407:x1871938519889152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.200824:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99860 00000100:00100000:3.0:1785220581.200846:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99861 00000100:00100000:3.0:1785220581.200852:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce79c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+530:2406:x1871938519889408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.200861:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.200866:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000070f37693/0xf62499ab92778426 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b766c2 expref: 530 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200876:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000070f37693/0xf62499ab92778426 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b766c2 expref: 530 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200888:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000070f37693/0xf62499ab92778426 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b766c2 expref: 530 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200898:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7152:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.200900:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7152:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.200904:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200911:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.200926:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.200944:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce79c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+529:2406:x1871938519889408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.200950:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99861 00000100:00100000:3.0:1785220581.200987:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99862 00000100:00100000:3.0:1785220581.200993:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+478:2406:x1871938519889280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.201001:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.201006:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003ebea4ba/0xf62499ab92780f23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b782c9 expref: 478 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201016:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003ebea4ba/0xf62499ab92780f23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b782c9 expref: 478 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201028:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ebea4ba/0xf62499ab92780f23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b782c9 expref: 478 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201036:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x718e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.201040:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x718e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.201043:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201055:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201072:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.201122:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+477:2406:x1871938519889280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 129us (683us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.201129:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99862 00000100:00100000:3.0:1785220581.201143:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99863 00000100:00100000:3.0:1785220581.201150:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+528:2406:x1871938519889536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.201158:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.201162:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ee9bd2d8/0xf62499ab92778140 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7151:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76660 expref: 528 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201172:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ee9bd2d8/0xf62499ab92778140 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7151:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76660 expref: 528 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201184:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7151:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.201186:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7151:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.201190:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4295 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7151:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201197:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4295 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7151:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201213:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.201229:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+528:2406:x1871938519889536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 80us (772us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.201235:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99863 00010000:00010000:3.0:1785220581.201253:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201261:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201269:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201277:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201290:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.201366:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1005:3655:x1871938510561024:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13507us (13659us total) trans 4295070377 rc 0/0 00000100:00100000:3.0:1785220581.201372:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283928 00000100:00100000:3.0:1785220581.201386:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283940 00000100:00100000:3.0:1785220581.201392:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1005:3655:x1871938510564480:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.201407:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29076 count 1 00010000:00010000:3.0:1785220581.201424:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a42e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201438:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a52f3cc3/0xf62499ab92781408 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7857e expref: 476 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201445:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7194:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.201448:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7194:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.201452:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a42e2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201472:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a52f3cc3/0xf62499ab92781408 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7857e expref: 476 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201484:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a52f3cc3/0xf62499ab92781408 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7857e expref: 476 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.201496:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0000_UUID:14202:1871938510564736:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.201522:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.201532:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.201537:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201544:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fee00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.201547:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201561:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201572:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201581:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a428e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201595:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.201597:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.201598:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.201607:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201615:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201624:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201631:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201645:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.201734:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+927:3652:x1871938510561792:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11672us (11704us total) trans 4295069779 rc 0/0 00000100:00100000:3.0:1785220581.201741:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283930 00010000:00010000:3.0:1785220581.201757:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.201762:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201782:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.201785:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201793:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201804:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201811:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a429c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7152:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.201827:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.201829:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.201831:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.202454:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.202460:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4280 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.202468:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.202472:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4280 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.202479:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4280 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.202491:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4280 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.202498:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4280 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.202512:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.202514:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.202515:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.203486:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283943 00000100:00100000:3.0:1785220581.203496:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+928:3652:x1871938510565120:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.203520:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29016 count 1 00010000:00010000:3.0:1785220581.203539:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a42f0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7158:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203553:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000031085c37/0xf62499ab92779368 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7158:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76970 expref: 526 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203562:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7158:0x0].0x0 since=38s v=(0 0) 00010000:00010000:3.0:1785220581.203566:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7158:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.203570:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a42f0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7158:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203595:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000031085c37/0xf62499ab92779368 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7158:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76970 expref: 526 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203670:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000031085c37/0xf62499ab92779368 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7158:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76970 expref: 526 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.203682:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510565376:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.203731:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.203746:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510565248 00000100:00100000:3.0:1785220581.203749:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510565248 00000100:00100000:3.0:1785220581.203781:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283944 00000100:00100000:3.0:1785220581.203787:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+929:3655:x1871938510565248:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.203798:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29017 count 1 00010000:00010000:3.0:1785220581.203809:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a42fe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203820:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fc29fd3a/0xf62499ab92779671 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a03 expref: 526 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203826:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7159:0x0].0x0 since=38s v=(0 0) 00010000:00010000:3.0:1785220581.203829:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7159:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.203832:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a42fe lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203861:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fc29fd3a/0xf62499ab92779671 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a03 expref: 526 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.203871:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fc29fd3a/0xf62499ab92779671 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a03 expref: 526 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.203880:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510565760:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.206966:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.207996:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97840e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510562816:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.208013:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a42aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214270:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.214277:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a42db lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7154:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214286:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.214290:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a42db lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7154:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214299:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a42db lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7154:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214313:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a42db lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7154:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214321:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a42db lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7154:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214339:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.214341:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.214343:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.214367:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0000_UUID:14202:1871938510564736:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.214389:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a42e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.214408:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99868 00000100:00100000:3.0:1785220581.214415:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+476:2406:x1871938519890048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.214426:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.214431:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000088d383eb/0xf62499ab9278128e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784b3 expref: 476 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214441:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000088d383eb/0xf62499ab9278128e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784b3 expref: 476 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214455:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000088d383eb/0xf62499ab9278128e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784b3 expref: 476 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214466:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7192:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.214469:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7192:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.214473:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214486:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214503:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.214519:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+475:2406:x1871938519890048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.214525:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99868 00010000:00010000:3.0:1785220581.214542:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214551:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214559:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214568:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.214582:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.214687:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1009:3653:x1871938510563200:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19034us (19642us total) trans 4295070380 rc 0/0 00000100:00100000:3.0:1785220581.214695:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283935 00000100:00100000:3.0:1785220581.214725:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510562944:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.214738:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a42b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.214756:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510565376:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.214779:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a42f0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7158:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.215002:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510565760:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.215017:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a42fe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216072:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.216079:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216105:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.216108:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216115:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216125:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216132:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a42bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216164:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.216166:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.216168:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.216179:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a42e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216188:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a42e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216197:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a42e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216205:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a42e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216219:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.216321:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1009:3655:x1871938510564480:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14929us (14979us total) trans 4295070381 rc 0/0 00000100:00100000:3.0:1785220581.216328:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283940 00010000:00010000:3.0:1785220581.216349:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a42b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216358:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a42b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216373:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a42b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216380:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a42b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.216459:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.216539:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1009:3653:x1871938510562688:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22141us (22609us total) trans 4295070382 rc 0/0 00000100:00100000:3.0:1785220581.216545:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283933 00000100:00100000:3.0:1785220581.216591:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.216642:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.216670:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.216686:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519891584 00000100:00100000:3.0:1785220581.216692:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519891584 00010000:00010000:3.0:1785220581.216884:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fc29fd3a/0xf62499ab92779671 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7159:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76a03 expref: 523 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.216903:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519891200 00000100:00100000:3.0:1785220581.216906:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519891200 00010000:00010000:3.0:1785220581.216921:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006f7293d1/0xf62499ab92779154 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7157:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76907 expref: 524 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.216934:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99875 00000100:00100000:3.0:1785220581.217661:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+524:2406:x1871938519890432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.217677:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.217683:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000afbd4ed4/0xf62499ab92778f78 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7156:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b768a5 expref: 524 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.217693:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000afbd4ed4/0xf62499ab92778f78 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7156:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b768a5 expref: 524 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.217707:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000afbd4ed4/0xf62499ab92778f78 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7156:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b768a5 expref: 524 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.217718:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7156:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.217721:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7156:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.217726:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a42d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7156:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.217734:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009a1704ba/0xf62499ab927a42d4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7156:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.217756:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.217796:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+523:2406:x1871938519890432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 854us (3180us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.217802:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99875 00000100:00100000:3.0:1785220581.217841:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99877 00000100:00100000:3.0:1785220581.219343:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+520:2407:x1871938519891200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.219354:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.219360:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006f7293d1/0xf62499ab92779154 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7157:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76907 expref: 520 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.219369:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006f7293d1/0xf62499ab92779154 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7157:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76907 expref: 520 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.219383:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f7293d1/0xf62499ab92779154 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7157:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76907 expref: 520 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.219393:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7157:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.219396:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7157:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.219400:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a42f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7157:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.219408:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a42f7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7157:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.219426:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.219448:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+519:2407:x1871938519891200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1601us (2777us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.219455:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99877 00000100:00100000:3.0:1785220581.219547:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.219628:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.219675:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.223910:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510568064 00000100:00100000:3.0:1785220581.223914:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510568064 00000100:00100000:3.0:1785220581.223935:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510567936 00000100:00100000:3.0:1785220581.223938:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510567936 00000100:00100000:3.0:1785220581.223953:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283951 00000100:00100000:3.0:1785220581.223960:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98adf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1011:3655:x1871938510566784:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.223984:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29080 count 1 00010000:00010000:3.0:1785220581.224004:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a432f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224023:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000072368fb2/0xf62499ab9278172d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78706 expref: 466 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224032:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7198:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.224036:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7198:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.224040:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a432f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224064:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000072368fb2/0xf62499ab9278172d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78706 expref: 466 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224077:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000072368fb2/0xf62499ab9278172d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78706 expref: 466 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.224103:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97840000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0000_UUID:14183:1871938510568832:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.224160:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.224174:0:14205:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510568448 00000100:00100000:3.0:1785220581.224177:0:14205:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510568448 00000100:00100000:3.0:1785220581.224382:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283953 00000100:00100000:3.0:1785220581.224390:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+938:3655:x1871938510568064:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.224401:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29024 count 1 00010000:00010000:3.0:1785220581.224412:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224423:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f9c17a99/0xf62499ab9277ad4d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d67 expref: 517 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224431:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7160:0x0].0x0 since=37s v=(0 0) 00010000:00010000:3.0:1785220581.224434:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7160:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.224437:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224454:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f9c17a99/0xf62499ab9277ad4d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d67 expref: 517 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224463:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f9c17a99/0xf62499ab9277ad4d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d67 expref: 517 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.224473:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510569088:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.224496:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.224513:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283955 00000100:00100000:3.0:1785220581.224520:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9147a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1012:3655:x1871938510567936:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.224532:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29083 count 1 00010000:00010000:3.0:1785220581.224542:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a434b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224558:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000047f1d14c/0xf62499ab927819a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78879 expref: 466 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224565:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x719b:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.224568:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.224572:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a434b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224588:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000047f1d14c/0xf62499ab927819a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78879 expref: 466 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224597:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000047f1d14c/0xf62499ab927819a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78879 expref: 466 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.224607:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a9b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510569344:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.224628:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.224643:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283957 00000100:00100000:3.0:1785220581.224650:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+938:3655:x1871938510568576:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.224660:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29026 count 1 00010000:00010000:3.0:1785220581.224670:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4359 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224681:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e3aa425b/0xf62499ab9277b2ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e63 expref: 517 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224694:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7162:0x0].0x0 since=37s v=(0 0) 00010000:00010000:3.0:1785220581.224697:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7162:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.224700:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4359 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224731:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e3aa425b/0xf62499ab9277b2ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e63 expref: 517 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224740:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e3aa425b/0xf62499ab9277b2ef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76e63 expref: 517 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.224749:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510569600:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.224780:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.224795:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283959 00000100:00100000:3.0:1785220581.224803:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1012:3655:x1871938510568448:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.224814:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29085 count 1 00010000:00010000:3.0:1785220581.224825:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224835:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019e4da63/0xf62499ab92781bb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7896e expref: 466 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224961:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x719d:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.224964:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.224968:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224987:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019e4da63/0xf62499ab92781bb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7896e expref: 466 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.224997:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019e4da63/0xf62499ab92781bb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7896e expref: 466 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.225008:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510569856:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.225027:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.225043:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283961 00000100:00100000:3.0:1785220581.225050:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1013:3652:x1871938510567808:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.225062:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29082 count 1 00010000:00010000:3.0:1785220581.225072:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4375 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225083:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a689aaf4/0xf62499ab92781910 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787fb expref: 466 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225239:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x719a:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.225242:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.225246:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4375 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225265:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a689aaf4/0xf62499ab92781910 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787fb expref: 466 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225276:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a689aaf4/0xf62499ab92781910 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787fb expref: 466 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.225286:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510570112:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.225309:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.225325:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510568320 00000100:00100000:3.0:1785220581.225328:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510568320 00000100:00100000:3.0:1785220581.225355:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283962 00000100:00100000:3.0:1785220581.225361:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1014:3652:x1871938510568320:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.225373:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29084 count 1 00010000:00010000:3.0:1785220581.225385:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225397:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f63b58fa/0xf62499ab92781a67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788f0 expref: 466 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225403:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x719c:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.225406:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.225408:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225426:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f63b58fa/0xf62499ab92781a67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788f0 expref: 466 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225438:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f63b58fa/0xf62499ab92781a67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788f0 expref: 466 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.225447:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510570240:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.225466:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.225602:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.225621:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519891968 00000100:00100000:3.0:1785220581.225625:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519891968 00010000:00010000:3.0:1785220581.225645:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000072278982/0xf62499ab9277a02d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x715c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b61 expref: 518 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.225665:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99882 00000100:00100000:3.0:1785220581.225671:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+518:2407:x1871938519891968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.225680:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.225685:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000072278982/0xf62499ab9277a02d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x715c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b61 expref: 518 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225696:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000072278982/0xf62499ab9277a02d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x715c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b61 expref: 518 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225716:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x715c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.225719:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x715c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.225724:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a4321 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x715c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225737:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a4321 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x715c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.225752:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.225777:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+518:2407:x1871938519891968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.225785:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99882 00000100:00100000:3.0:1785220581.226040:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.234282:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510569088:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.234301:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.234323:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a9b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510569344:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.234334:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a434b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.234354:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510569856:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.234363:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.234383:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510570112:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.234392:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4375 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.234415:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510570240:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.234425:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.234443:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510569600:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.234452:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4359 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.234486:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519892480 00000100:00100000:3.0:1785220581.234489:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519892480 00010000:00010000:3.0:1785220581.234502:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f9c17a99/0xf62499ab9277ad4d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d67 expref: 518 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.236749:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519893248 00000100:00100000:3.0:1785220581.236754:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519893248 00010000:00010000:3.0:1785220581.236783:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a689aaf4/0xf62499ab92781910 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787fb expref: 469 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.236795:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519893504 00000100:00100000:3.0:1785220581.236797:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519893504 00010000:00010000:3.0:1785220581.236809:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000069176708/0xf62499ab9277a96b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x715f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76cbf expref: 522 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.236821:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99885 00000100:00100000:3.0:1785220581.236828:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+522:2405:x1871938519892352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.236837:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.236900:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009b638d97/0xf62499ab9277a5d6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x715e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76c3a expref: 522 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.236910:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009b638d97/0xf62499ab9277a5d6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x715e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76c3a expref: 522 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.236926:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009b638d97/0xf62499ab9277a5d6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x715e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76c3a expref: 522 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.236936:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x715e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.236939:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x715e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.236943:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4336 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x715e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.236951:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4336 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x715e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.236969:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.237000:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90140e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+521:2405:x1871938519892352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 172us (3274us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.237008:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99885 00000100:00100000:3.0:1785220581.237028:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99886 00000100:00100000:3.0:1785220581.237036:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+520:2405:x1871938519892480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.237046:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.237051:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f9c17a99/0xf62499ab9277ad4d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d67 expref: 520 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237061:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f9c17a99/0xf62499ab9277ad4d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d67 expref: 520 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237074:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f9c17a99/0xf62499ab9277ad4d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76d67 expref: 520 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237238:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7160:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237243:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7160:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237248:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237256:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237275:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.237329:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+519:2405:x1871938519892480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 293us (3582us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.237336:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99886 00000100:00100000:3.0:1785220581.237475:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99888 00000100:00100000:3.0:1785220581.237482:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+518:2405:x1871938519892864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.237511:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.237517:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000edb477be/0xf62499ab9277b1bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7161:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76df3 expref: 518 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237526:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000edb477be/0xf62499ab9277b1bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7161:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76df3 expref: 518 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237537:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edb477be/0xf62499ab9277b1bb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7161:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76df3 expref: 518 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237547:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7161:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237550:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7161:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237554:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4360 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7161:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237562:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4360 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7161:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237575:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.237592:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+516:2405:x1871938519892864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (3770us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.237599:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99888 00000100:00100000:3.0:1785220581.237614:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99890 00000100:00100000:3.0:1785220581.237620:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+467:2405:x1871938519893120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.237628:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.237632:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019e4da63/0xf62499ab92781bb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7896e expref: 467 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237647:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019e4da63/0xf62499ab92781bb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7896e expref: 467 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237660:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019e4da63/0xf62499ab92781bb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7896e expref: 467 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237671:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237674:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237678:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237687:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237703:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.237719:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+465:2405:x1871938519893120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (3860us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.237726:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99890 00000100:00100000:3.0:1785220581.237738:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99892 00000100:00100000:3.0:1785220581.237744:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+464:2405:x1871938519893376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.237752:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.237757:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f63b58fa/0xf62499ab92781a67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788f0 expref: 463 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237787:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f63b58fa/0xf62499ab92781a67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788f0 expref: 463 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237799:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63b58fa/0xf62499ab92781a67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788f0 expref: 463 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237809:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237812:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.237817:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237825:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237910:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.237926:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+462:2405:x1871938519893376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 181us (4035us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.237932:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99892 00010000:00010000:3.0:1785220581.237952:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237960:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237967:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237973:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a433d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7160:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.237986:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.238345:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+941:3655:x1871938510568064:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13957us (15014us total) trans 4295069791 rc 0/0 00000100:00100000:3.0:1785220581.238352:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283953 00010000:00010000:3.0:1785220581.238373:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a434b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238383:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a434b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238391:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a434b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238398:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a434b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238410:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.238483:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1018:3655:x1871938510567936:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13963us (15057us total) trans 4295070390 rc 0/0 00000100:00100000:3.0:1785220581.238489:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283955 00010000:00010000:3.0:1785220581.238509:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238523:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238533:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238540:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4367 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238561:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.238626:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1018:3655:x1871938510568448:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13825us (14962us total) trans 4295070391 rc 0/0 00000100:00100000:3.0:1785220581.238633:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283959 00010000:00010000:3.0:1785220581.238655:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238664:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238672:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238679:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a437c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.238700:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.238783:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1018:3652:x1871938510568320:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13423us (13598us total) trans 4295070392 rc 0/0 00000100:00100000:3.0:1785220581.238790:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283962 00000100:00100000:3.0:1785220581.238945:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510571392 00000100:00100000:3.0:1785220581.238950:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510571392 00000100:00100000:3.0:1785220581.238978:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283967 00000100:00100000:3.0:1785220581.238985:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+941:3653:x1871938510571392:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.239006:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29029 count 1 00010000:00010000:3.0:1785220581.239024:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a439f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239039:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c5b667b8/0xf62499ab9277ba43 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76f97 expref: 510 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239047:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7165:0x0].0x0 since=37s v=(0 0) 00010000:00010000:3.0:1785220581.239050:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7165:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.239054:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a439f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239075:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c5b667b8/0xf62499ab9277ba43 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76f97 expref: 510 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239275:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c5b667b8/0xf62499ab9277ba43 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76f97 expref: 510 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.239295:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510571776:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.239335:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.239348:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510571648 00000100:00100000:3.0:1785220581.239352:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510571648 00000100:00100000:3.0:1785220581.239374:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510571520 00000100:00100000:3.0:1785220581.239376:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510571520 00000100:00100000:3.0:1785220581.239394:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283968 00000100:00100000:3.0:1785220581.239399:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1019:3655:x1871938510571648:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.239411:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29090 count 1 00010000:00010000:3.0:1785220581.239423:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a43a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239436:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005e0d54a6/0xf62499ab92782128 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78beb expref: 460 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239444:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71a2:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.239447:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71a2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.239451:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a43a6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239470:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005e0d54a6/0xf62499ab92782128 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78beb expref: 460 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239481:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005e0d54a6/0xf62499ab92782128 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78beb expref: 460 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.239495:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510571904:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.239524:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.239568:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.239589:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519893760 00000100:00100000:3.0:1785220581.239592:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519893760 00010000:00010000:3.0:1785220581.239608:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005d5f9aea/0xf62499ab92781c82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b789ec expref: 461 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.239627:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99895 00000100:00100000:3.0:1785220581.239633:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+461:2406:x1871938519893760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.239642:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.239647:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005d5f9aea/0xf62499ab92781c82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x719e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b789ec expref: 461 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239658:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005d5f9aea/0xf62499ab92781c82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x719e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b789ec expref: 461 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239672:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d5f9aea/0xf62499ab92781c82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x719e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b789ec expref: 461 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239681:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.239684:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x719e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.239690:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a4383 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239702:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a4383 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.239718:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.239732:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+460:2406:x1871938519893760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (165us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.239738:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99895 00000100:00100000:3.0:1785220581.239829:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.239943:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.240026:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.240715:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.240911:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519894016 00000100:00100000:3.0:1785220581.240915:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519894016 00010000:00010000:3.0:1785220581.240930:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005118bc74/0xf62499ab92781e9d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ab7 expref: 461 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.241004:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99898 00000100:00100000:3.0:1785220581.241010:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+512:2406:x1871938519894144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.241018:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.241023:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c86725d0/0xf62499ab9277b7aa lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7164:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76f27 expref: 512 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.241036:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c86725d0/0xf62499ab9277b7aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7164:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76f27 expref: 512 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.241054:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7164:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.241058:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7164:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.241063:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4391 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7164:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.241072:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4391 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7164:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.241246:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.241267:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+512:2406:x1871938519894144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 257us (552us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.241274:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99898 00010000:00010000:3.0:1785220581.244392:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4359 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244401:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4359 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244409:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4359 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244416:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4359 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7162:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244429:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.244555:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+944:3655:x1871938510568576:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19905us (20803us total) trans 4295069798 rc 0/0 00000100:00100000:3.0:1785220581.244562:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283957 00000100:00100000:3.0:1785220581.244578:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510572672 00000100:00100000:3.0:1785220581.244582:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510572672 00000100:00100000:3.0:1785220581.244610:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283972 00000100:00100000:3.0:1785220581.244617:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90142680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+944:3655:x1871938510572672:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.244635:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29032 count 1 00010000:00010000:3.0:1785220581.244652:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244666:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008beba819/0xf62499ab9277ca65 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77149 expref: 508 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244674:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7168:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.244677:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7168:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.244681:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244703:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008beba819/0xf62499ab9277ca65 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77149 expref: 508 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.244714:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008beba819/0xf62499ab9277ca65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77149 expref: 508 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.244725:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510572800:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.244996:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.247915:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.248369:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510573184 00000100:00100000:3.0:1785220581.248374:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510573184 00000100:00100000:3.0:1785220581.248680:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283976 00000100:00100000:3.0:1785220581.248688:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+946:3653:x1871938510573696:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.248701:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29034 count 1 00010000:00010000:3.0:1785220581.248714:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a43de lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.248726:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a876579c/0xf62499ab9277d15e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77237 expref: 508 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.248733:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x716a:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.248736:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x716a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.248740:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a43de lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.248757:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a876579c/0xf62499ab9277d15e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77237 expref: 508 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.248781:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a876579c/0xf62499ab9277d15e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77237 expref: 508 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.248792:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510574592:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.251598:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.251639:0:14256:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e0d54a6/0xf62499ab92782128 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78beb expref: 456 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.251649:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510571904:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.251667:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a43a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251676:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a43a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251683:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a43a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251691:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a43a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251699:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a43a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251720:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.251837:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1024:3655:x1871938510571648:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12437us (12845us total) trans 4295070397 rc 0/0 00000100:00100000:3.0:1785220581.251844:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283968 00010000:00010000:3.0:1785220581.251889:0:14189:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c5b667b8/0xf62499ab9277ba43 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76f97 expref: 506 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.251906:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510571776:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.251918:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a439f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251926:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a439f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251933:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a439f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251941:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a439f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251947:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a439f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7165:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.251961:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.252032:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+947:3653:x1871938510571392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13047us (13104us total) trans 4295069801 rc 0/0 00000100:00100000:3.0:1785220581.252039:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283967 00010000:00010000:3.0:1785220581.252071:0:14262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008beba819/0xf62499ab9277ca65 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77149 expref: 505 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.252081:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510572800:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.252125:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252133:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252141:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252149:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252157:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252175:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.252244:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90142680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+947:3655:x1871938510572672:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7626us (7749us total) trans 4295069802 rc 0/0 00000100:00100000:3.0:1785220581.252250:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283972 00010000:00010000:3.0:1785220581.252267:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.252272:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252280:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.252283:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252297:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252309:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252317:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a43c2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7168:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.252331:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.252333:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.252335:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.253848:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283984 00000100:00100000:3.0:1785220581.253858:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+951:3655:x1871938510575232:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.253871:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29039 count 1 00010000:00010000:3.0:1785220581.253883:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4416 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.253895:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000072a7cead/0xf62499ab9277e2ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77460 expref: 504 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.253902:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x716f:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.253904:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x716f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.253908:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4416 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.253931:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000072a7cead/0xf62499ab9277e2ec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77460 expref: 504 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.253941:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000072a7cead/0xf62499ab9277e2ec lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77460 expref: 504 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.253951:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510576384:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.254460:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.257222:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283986 00000100:00100000:3.0:1785220581.257238:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d987e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1027:3654:x1871938510573312:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.257272:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29093 count 1 00010000:00010000:3.0:1785220581.257299:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257320:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ce29c949/0xf62499ab927823cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d7a expref: 455 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257330:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71a5:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.257335:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.257340:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257375:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ce29c949/0xf62499ab927823cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d7a expref: 455 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257396:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ce29c949/0xf62499ab927823cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d7a expref: 455 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.257411:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0000_UUID:14202:1871938510576640:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.257481:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.257510:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283987 00000100:00100000:3.0:1785220581.257519:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfa5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+952:3654:x1871938510573824:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.257535:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29035 count 1 00010000:00010000:3.0:1785220581.257549:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257562:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006654fa4f/0xf62499ab9277d754 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b772a0 expref: 504 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257570:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x716b:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.257574:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x716b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.257577:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257599:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006654fa4f/0xf62499ab9277d754 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b772a0 expref: 504 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.257609:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006654fa4f/0xf62499ab9277d754 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b772a0 expref: 504 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.257626:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510576768:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.257650:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.258930:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510574080:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.258943:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a43c9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.259055:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99906 00000100:00100000:3.0:1785220581.259064:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+505:2406:x1871938519895040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.259076:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.259082:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001724bc12/0xf62499ab9277cea9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b771c7 expref: 505 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259117:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001724bc12/0xf62499ab9277cea9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b771c7 expref: 505 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259138:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001724bc12/0xf62499ab9277cea9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b771c7 expref: 505 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259153:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7169:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.259157:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7169:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.259163:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a43c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259177:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a43c9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259193:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.259529:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+504:2406:x1871938519895040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 466us (704us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.259536:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99906 00010000:00010000:3.0:1785220581.259558:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a43c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259568:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a43c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259578:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a43c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259588:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a43c9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7169:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.259823:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.265290:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+953:3652:x1871938510573056:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12570us (13616us total) trans 4295069803 rc 0/0 00000100:00100000:3.0:1785220581.265301:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283973 00010000:00010000:3.0:1785220581.265470:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.265494:0:14164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000072a7cead/0xf62499ab9277e2ec lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77460 expref: 500 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.265506:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510576384:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.265524:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4416 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.265533:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4416 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.265540:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4416 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.265548:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4416 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.265556:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4416 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.265568:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.265637:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+953:3655:x1871938510575232:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11782us (12040us total) trans 4295069807 rc 0/0 00000100:00100000:3.0:1785220581.265644:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283984 00000100:00100000:3.0:1785220581.266054:0:14164:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510578176 00000100:00100000:3.0:1785220581.266058:0:14164:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510578176 00010000:00010000:2.0:1785220581.267539:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7170:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.267547:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267557:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267585:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.267628:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+500:2407:x1871938519896576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 163us (941us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.267635:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99918 00000100:00100000:2.0:1785220581.267649:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99919 00000100:00100000:2.0:1785220581.267655:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+499:2407:x1871938519896832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.267662:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.267667:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006654fa4f/0xf62499ab9277d754 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b772a0 expref: 499 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267677:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006654fa4f/0xf62499ab9277d754 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b772a0 expref: 499 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267689:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x716b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.267692:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x716b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.267697:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267705:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267722:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.267739:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+499:2407:x1871938519896832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (993us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.267746:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99919 00000100:00100000:2.0:1785220581.267758:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99920 00000100:00100000:2.0:1785220581.267776:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+450:2407:x1871938519896704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.267785:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.267790:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fe5ad4f7/0xf62499ab927827f0 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f9c expref: 450 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267800:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fe5ad4f7/0xf62499ab927827f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f9c expref: 450 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267814:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.267817:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.267822:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a4432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267829:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a4432 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267852:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.267870:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+450:2407:x1871938519896704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (1105us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.267881:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99920 00010000:00010000:2.0:1785220581.267906:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267916:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267926:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267933:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.267957:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.268148:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+953:3653:x1871938510575360:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14445us (14734us total) trans 4295069808 rc 0/0 00000100:00100000:2.0:1785220581.268157:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283982 00010000:00010000:2.0:1785220581.268176:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.268182:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.268190:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.268193:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.268202:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.268214:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.268905:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a4408 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7170:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.268924:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.269051:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.269053:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.269081:0:14190:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe5ad4f7/0xf62499ab927827f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f9c expref: 449 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.269148:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283994 00000100:00100000:3.0:1785220581.269161:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1033:3653:x1871938510578176:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.269178:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29104 count 1 00010000:00010000:3.0:1785220581.269191:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269203:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008bd87dfe/0xf62499ab92782d30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79204 expref: 449 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269211:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b0:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.269214:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.269218:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269239:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008bd87dfe/0xf62499ab92782d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79204 expref: 449 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269248:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008bd87dfe/0xf62499ab92782d30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79204 expref: 449 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.269258:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510578432:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.269284:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.269312:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0000_UUID:14202:1871938510576640:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.269331:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269346:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.269374:0:14208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a876579c/0xf62499ab9277d15e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77237 expref: 498 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.269374:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510576896:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.269385:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510574592:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.269388:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a4432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269397:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a43de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269397:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a4432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269406:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a4432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269407:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a43de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269414:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a43de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269417:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a4432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269422:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a43de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.269430:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a43de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269431:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a4432 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269474:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.269604:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1034:3655:x1871938510575872:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11595us (13687us total) trans 4295070405 rc 0/0 00000100:00100000:2.0:1785220581.269611:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283988 00000100:00100000:2.0:1785220581.269633:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519896960 00000100:00100000:2.0:1785220581.269640:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519896960 00000100:00100000:2.0:1785220581.269659:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510577536:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.269671:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269698:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ce29c949/0xf62499ab927823cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d7a expref: 449 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.269724:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99921 00000100:00100000:2.0:1785220581.269730:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+449:2406:x1871938519896960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.269738:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.269743:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ce29c949/0xf62499ab927823cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d7a expref: 449 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269754:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ce29c949/0xf62499ab927823cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d7a expref: 449 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269791:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce29c949/0xf62499ab927823cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d7a expref: 449 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269802:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.269805:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.269810:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269818:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269845:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.269864:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+448:2406:x1871938519896960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 135us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.269870:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99921 00010000:00010000:2.0:1785220581.269886:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269895:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269903:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.269911:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270401:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.270514:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+953:3653:x1871938510573696:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21827us (22434us total) trans 4295069809 rc 0/0 00000100:00100000:3.0:1785220581.270521:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283976 00000100:00100000:3.0:1785220581.270538:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510578560 00000100:00100000:3.0:1785220581.270543:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510578560 00000100:00100000:3.0:1785220581.270575:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283995 00000100:00100000:3.0:1785220581.270582:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1034:3653:x1871938510578560:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.270599:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29105 count 1 00010000:00010000:3.0:1785220581.270616:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270629:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000af06cda0/0xf62499ab92782e1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79289 expref: 447 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270637:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b1:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.270640:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.270644:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270664:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000af06cda0/0xf62499ab92782e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79289 expref: 447 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270674:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000af06cda0/0xf62499ab92782e1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79289 expref: 447 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.270686:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510578688:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.270730:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.270747:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519897088 00000100:00100000:3.0:1785220581.270750:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519897088 00010000:00010000:3.0:1785220581.270777:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b383cd8b/0xf62499ab92782aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b790c9 expref: 448 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.270799:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519897344 00000100:00100000:3.0:1785220581.270802:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519897344 00010000:00010000:3.0:1785220581.270816:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d1785aea/0xf62499ab9278290f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7900c expref: 449 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.270833:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519897216 00000100:00100000:3.0:1785220581.270848:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519897216 00010000:00010000:3.0:1785220581.270862:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000dd1f996/0xf62499ab9278297f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7903d expref: 450 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.270886:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99922 00000100:00100000:3.0:1785220581.270893:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+450:2407:x1871938519897088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.270902:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.270906:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b383cd8b/0xf62499ab92782aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b790c9 expref: 450 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270915:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b383cd8b/0xf62499ab92782aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b790c9 expref: 450 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270931:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b383cd8b/0xf62499ab92782aa5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b790c9 expref: 450 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270943:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.270946:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.270950:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270959:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.270980:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.271003:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+449:2407:x1871938519897088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (1102us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.271010:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99922 00000100:00100000:3.0:1785220581.271027:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99923 00000100:00100000:3.0:1785220581.271033:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+448:2407:x1871938519897344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.271041:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.271045:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d1785aea/0xf62499ab9278290f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7900c expref: 448 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271054:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d1785aea/0xf62499ab9278290f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7900c expref: 448 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271065:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d1785aea/0xf62499ab9278290f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7900c expref: 448 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271074:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.271076:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.271080:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271110:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271135:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.271154:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+447:2407:x1871938519897344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 121us (1232us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.271161:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99923 00000100:00100000:3.0:1785220581.271175:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99924 00000100:00100000:3.0:1785220581.271181:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+446:2407:x1871938519897216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.271189:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.271193:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000dd1f996/0xf62499ab9278297f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7903d expref: 446 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271203:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000dd1f996/0xf62499ab9278297f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7903d expref: 446 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271215:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dd1f996/0xf62499ab9278297f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7903d expref: 446 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271225:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.271227:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.271231:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271238:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271260:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.271277:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+445:2407:x1871938519897216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1030us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.271284:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99924 00010000:00010000:3.0:1785220581.271323:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271334:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271343:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.271352:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.273826:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.273830:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.273837:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.273844:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.273851:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.273852:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.273854:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.274325:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90142300/2, rc 0, transno 4295069809 00000100:00100000:0.0:1785220581.274335:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90142300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510573696:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.274583:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220581.274592:0:14195:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4447 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.274807:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+1035:3653:x1871938510577152:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13375us (13915us total) trans 4295070406 rc 0/0 00000100:00100000:3.0:1785220581.274818:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283991 00010000:00010000:3.0:1785220581.274868:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.274883:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.274894:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.274902:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.274916:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.274986:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1035:3653:x1871938510577280:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13978us (14138us total) trans 4295070407 rc 0/0 00000100:00100000:3.0:1785220581.274993:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283990 00010000:00010000:3.0:1785220581.275651:0:14250:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006654fa4f/0xf62499ab9277d754 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b772a0 expref: 497 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.275665:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510576768:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.275690:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.275701:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.275709:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.275717:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.275724:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.275734:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220581.275945:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.275953:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.275961:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.275965:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.275974:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.275991:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276002:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4440 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276020:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.276023:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276030:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:3.0:1785220581.276031:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfa5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+953:3654:x1871938510573824:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18514us (20689us total) trans 4295069810 rc 0/0 00010000:00010000:0.0:1785220581.276033:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.276038:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283987 00010000:00010000:0.0:1785220581.276041:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276050:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276057:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a442b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x716b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276065:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276072:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276072:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.276073:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.276075:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.276080:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.276081:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.276122:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510579200:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.276173:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276190:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.276268:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1035:3654:x1871938510577024:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15274us (15511us total) trans 4295070408 rc 0/0 00000100:00100000:3.0:1785220581.276278:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283989 00010000:00010000:3.0:1785220581.276296:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.276300:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276307:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.276310:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276318:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276329:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276337:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4439 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276352:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.276354:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.276355:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.276367:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.276403:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510579200 00000100:00100000:0.0:1785220581.276408:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510579200 00000100:00100000:0.0:1785220581.276444:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283996 00000100:00100000:0.0:1785220581.276451:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+953:3653:x1871938510579200:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.276474:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29044 count 1 00010000:00010000:0.0:1785220581.276494:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276509:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002a524c40/0xf62499ab9277efdb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77651 expref: 496 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276518:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7174:0x0].0x0 since=36s v=(0 0) 00010000:00010000:0.0:1785220581.276522:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7174:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.276527:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276552:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002a524c40/0xf62499ab9277efdb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77651 expref: 496 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276565:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002a524c40/0xf62499ab9277efdb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77651 expref: 496 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.276581:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510579712:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.276612:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510578944 00000100:00100000:3.0:1785220581.276616:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510578944 00000100:00100000:0.0:1785220581.276637:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f900 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.276638:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510579456 00000100:00100000:3.0:1785220581.276640:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510579456 00000100:00100000:0.0:1785220581.276652:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283997 00000100:00100000:3.0:1785220581.276656:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283998 00000100:00100000:0.0:1785220581.276659:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+955:3655:x1871938510578944:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.276661:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+955:3655:x1871938510579456:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.276671:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29046 count 1 00002000:00080000:0.0:1785220581.276671:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29042 count 1 00010000:00010000:3.0:1785220581.276683:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276683:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276696:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e5bd3942/0xf62499ab9277f5a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77746 expref: 496 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276696:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e1d7ad8b/0xf62499ab9277ea16 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775b0 expref: 496 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276703:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7172:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.276704:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7176:0x0].0x0 since=36s v=(0 0) 00010000:00010000:3.0:1785220581.276707:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7176:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.276707:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7172:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.276712:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276712:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276730:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e5bd3942/0xf62499ab9277f5a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77746 expref: 496 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276730:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e1d7ad8b/0xf62499ab9277ea16 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775b0 expref: 496 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.276741:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e5bd3942/0xf62499ab9277f5a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77746 expref: 496 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.276741:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e1d7ad8b/0xf62499ab9277ea16 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775b0 expref: 496 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.276752:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510579968:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.276752:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510579840:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.276825:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1785220581.276832:0:14202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a4424 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.276866:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.276918:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a76d180 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.276957:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d987e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1035:3654:x1871938510573312:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19725us (21782us total) trans 4295070409 rc 0/0 00000100:00100000:2.0:1785220581.276965:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283986 00000100:00100000:0.0:1785220581.277008:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fcc0 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.277036:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365ed80/2, rc 0, transno 4295070406 00000100:00100000:0.0:1785220581.277050:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510577152:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.277062:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365e300/2, rc 0, transno 4295070407 00000100:00100000:0.0:1785220581.277065:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510577280:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.277079:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1785220581.277110:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510579072:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.277489:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510577920:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.277513:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.277585:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.277606:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.277625:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.277671:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.277936:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510578816 00000100:00100000:2.0:1785220581.277941:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510578816 00000100:00100000:2.0:1785220581.277968:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283999 00000100:00100000:2.0:1785220581.277975:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+956:3655:x1871938510578816:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.277995:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29041 count 1 00010000:00010000:2.0:1785220581.278013:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.278027:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e5b9f022/0xf62499ab9277e825 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77524 expref: 496 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.278036:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7171:0x0].0x0 since=36s v=(0 0) 00010000:00010000:2.0:1785220581.278039:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7171:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.278044:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.278069:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e5b9f022/0xf62499ab9277e825 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77524 expref: 496 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.278080:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e5b9f022/0xf62499ab9277e825 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77524 expref: 496 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.278691:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510578304:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.278712:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.278740:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220581.278753:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510580224:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.279336:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.279463:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284001 00000100:00100000:0.0:1785220581.279470:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1035:3652:x1871938510580224:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.279482:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29106 count 1 00010000:00010000:0.0:1785220581.279500:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.279514:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000084f15ec0/0xf62499ab92782ecd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792dd expref: 445 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.279528:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b2:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.279531:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.279535:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.279555:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000084f15ec0/0xf62499ab92782ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792dd expref: 445 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.279567:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000084f15ec0/0xf62499ab92782ecd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792dd expref: 445 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.279580:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510580480:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.279636:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.279724:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510580096:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.279828:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510578432:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.279856:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.279883:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510578688:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.279894:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.280268:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519897472 00000100:00100000:3.0:1785220581.280274:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519897472 00010000:00010000:3.0:1785220581.280299:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000dbb0bdc/0xf62499ab92782ca4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791a9 expref: 446 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.280329:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519897728 00000100:00100000:3.0:1785220581.280335:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519897728 00010000:00010000:3.0:1785220581.280351:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008807264d/0xf62499ab92782b93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79132 expref: 447 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.280370:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519897856 00000100:00100000:3.0:1785220581.280376:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519897856 00010000:00010000:3.0:1785220581.280391:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000af06cda0/0xf62499ab92782e1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79289 expref: 448 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.280415:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99925 00000100:00100000:3.0:1785220581.280422:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+448:2407:x1871938519897600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.280439:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.280444:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008bd87dfe/0xf62499ab92782d30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79204 expref: 448 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.280458:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008bd87dfe/0xf62499ab92782d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79204 expref: 448 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.280489:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd87dfe/0xf62499ab92782d30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79204 expref: 448 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.280506:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.280512:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.280517:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.280528:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.280554:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.282374:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510579712:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.282398:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.282447:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510579840:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.282464:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.282493:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.282512:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519898112 00000100:00100000:0.0:1785220581.282519:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519898112 00010000:00010000:0.0:1785220581.282542:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002a524c40/0xf62499ab9277efdb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77651 expref: 497 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.282567:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99927 00000100:00100000:0.0:1785220581.282574:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+446:2407:x1871938519897728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.282581:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.282586:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008807264d/0xf62499ab92782b93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79132 expref: 446 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282596:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008807264d/0xf62499ab92782b93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79132 expref: 446 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282608:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008807264d/0xf62499ab92782b93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79132 expref: 446 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282618:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.282621:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.282624:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282631:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282647:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.282661:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+445:2407:x1871938519897728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 89us (5036us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.282668:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99927 00000100:00100000:0.0:1785220581.282679:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99929 00000100:00100000:0.0:1785220581.282684:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+497:2406:x1871938519898112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.282687:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.282691:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.282693:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000af06cda0/0xf62499ab92782e1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79289 expref: 443 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282695:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002a524c40/0xf62499ab9277efdb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77651 expref: 497 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282704:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000af06cda0/0xf62499ab92782e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79289 expref: 443 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282706:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002a524c40/0xf62499ab9277efdb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77651 expref: 497 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282717:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af06cda0/0xf62499ab92782e1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79289 expref: 443 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282720:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a524c40/0xf62499ab9277efdb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77651 expref: 497 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282728:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.282729:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7174:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.282731:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.282732:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7174:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.282735:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282736:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282745:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.282745:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282760:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.282760:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.282791:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+442:2407:x1871938519897856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (5120us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.282797:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99928 00010000:00010000:1.0:1785220581.282818:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282826:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282840:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282848:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.282865:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.283022:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1036:3655:x1871938510578048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14423us (17217us total) trans 4295070410 rc 0/0 00000100:00100000:1.0:1785220581.283030:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283993 00010000:00010000:1.0:1785220581.283051:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.283057:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283065:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.283069:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283079:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.283080:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+496:2406:x1871938519898112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 394us (586us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.283166:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99929 00010000:00010000:0.0:1785220581.283185:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283194:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283202:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283209:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283288:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283295:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.283296:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4455 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283314:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.283317:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283324:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.283327:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283334:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283345:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283352:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a446a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7174:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.283367:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.283369:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.283370:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.283387:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+958:3653:x1871938510579200:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6936us (7019us total) trans 4295069811 rc 0/0 00000100:00100000:1.0:1785220581.283391:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.283393:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283996 00000100:00100000:0.0:1785220581.283407:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510579072 00000100:00100000:0.0:1785220581.283412:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510579072 00000100:00100000:1.0:1785220581.283427:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510579584:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.283441:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284002 00000100:00100000:0.0:1785220581.283448:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+958:3653:x1871938510579072:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.283461:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:0.0:1785220581.283469:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29043 count 1 00010000:00010000:0.0:1785220581.283486:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.283488:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a4000/2, rc 0, transno 4295069811 00000100:00100000:1.0:1785220581.283499:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510579200:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.283501:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006d3bdb01/0xf62499ab9277eb89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775fd expref: 495 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283513:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7173:0x0].0x0 since=36s v=(0 0) 00010000:00010000:0.0:1785220581.283515:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7173:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.283519:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283538:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006d3bdb01/0xf62499ab9277eb89 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775fd expref: 495 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283549:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006d3bdb01/0xf62499ab9277eb89 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775fd expref: 495 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.283560:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510580864:192.168.201.51@tcp:104: 00000004:00080000:1.0:1785220581.283565:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbfb80/2, rc 0, transno 4295070410 00000100:00100000:1.0:1785220581.283574:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510578048:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.283594:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f900 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.283604:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.283607:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.283614:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510580992:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.283614:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283622:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283631:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.283636:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.283639:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283658:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.283740:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1036:3655:x1871938510577792:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19109us (19220us total) trans 4295070411 rc 0/0 00000100:00100000:0.0:1785220581.283747:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283992 00000100:00100000:0.0:1785220581.283759:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510579584 00000100:00100000:0.0:1785220581.283762:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510579584 00000100:00100000:0.0:1785220581.283793:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510580992 00000100:00100000:0.0:1785220581.283796:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510580992 00000100:00100000:0.0:1785220581.283815:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284003 00000100:00100000:0.0:1785220581.283821:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+959:3653:x1871938510579584:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.283832:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29047 count 1 00010000:00010000:0.0:1785220581.283905:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283917:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a1c71f47/0xf62499ab9277f664 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b777af expref: 495 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283923:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7177:0x0].0x0 since=35s v=(0 0) 00010000:00010000:0.0:1785220581.283926:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7177:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.283930:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283948:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a1c71f47/0xf62499ab9277f664 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b777af expref: 495 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.283956:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a1c71f47/0xf62499ab9277f664 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b777af expref: 495 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.283966:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510581120:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.283987:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.284003:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284004 00000100:00100000:0.0:1785220581.284011:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1036:3653:x1871938510580992:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.284023:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29108 count 1 00010000:00010000:0.0:1785220581.284035:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.284047:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005db1b4c9/0xf62499ab9278308d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793bd expref: 441 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.284054:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b4:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.284057:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.284060:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.284078:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005db1b4c9/0xf62499ab9278308d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793bd expref: 441 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.285286:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005db1b4c9/0xf62499ab9278308d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793bd expref: 441 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.285310:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510581248:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.285375:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.285410:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.285417:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285425:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.285428:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.285434:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.285438:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285455:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.285459:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519897984 00010000:00010000:1.0:1785220581.285462:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a444e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.285465:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519897984 00010000:00010000:1.0:1785220581.285484:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.285486:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.285487:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.285490:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e5bd3942/0xf62499ab9277f5a7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77746 expref: 496 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.285515:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99930 00000004:00080000:1.0:1785220581.285521:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbca80/2, rc 0, transno 4295070411 00000100:00100000:0.0:1785220581.285523:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+496:2406:x1871938519897984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.285531:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510577792:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.285533:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.285538:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e5bd3942/0xf62499ab9277f5a7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77746 expref: 496 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.285547:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e5bd3942/0xf62499ab9277f5a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77746 expref: 496 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.285563:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7176:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.285566:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7176:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.285571:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.285579:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.285581:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510581376:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.285598:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.285610:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.285618:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+496:2406:x1871938519897984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (184us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.285626:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99930 00000100:00100000:1.0:1785220581.285644:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510580736:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.285668:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.285689:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510581376 00000100:00100000:1.0:1785220581.285692:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510581376 00000100:00100000:0.0:1785220581.285696:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.285713:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510580736 00000100:00100000:1.0:1785220581.285716:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510580736 00000100:00100000:1.0:1785220581.285733:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284005 00000100:00100000:1.0:1785220581.285738:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1037:3653:x1871938510581376:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.285756:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29109 count 1 00010000:00010000:1.0:1785220581.285785:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285798:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d2af55e1/0xf62499ab927831b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7942d expref: 441 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285805:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b5:0x0].0x0 since=32s v=(0 0) 00010000:00010000:1.0:1785220581.285808:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.285812:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285832:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d2af55e1/0xf62499ab927831b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7942d expref: 441 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285848:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d2af55e1/0xf62499ab927831b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7942d expref: 441 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.285859:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510581504:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.285883:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.285899:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284006 00000100:00100000:1.0:1785220581.285907:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+960:3655:x1871938510580736:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.285920:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29048 count 1 00010000:00010000:1.0:1785220581.285933:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285946:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000540330bf/0xf62499ab9277f760 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7782d expref: 495 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285952:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7178:0x0].0x0 since=35s v=(0 0) 00010000:00010000:1.0:1785220581.285955:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7178:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.285958:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285978:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000540330bf/0xf62499ab9277f760 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7782d expref: 495 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.285988:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000540330bf/0xf62499ab9277f760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7782d expref: 495 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.285997:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510581632:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.286022:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01b40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.286042:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.286044:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519898240 00000100:00100000:2.0:1785220581.286048:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519898240 00000100:00100000:1.0:1785220581.286048:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510580352:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.286067:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e1d7ad8b/0xf62499ab9277ea16 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775b0 expref: 496 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.286070:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.286123:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99931 00000100:00100000:2.0:1785220581.286131:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+496:2407:x1871938519898240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.286195:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.286201:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e1d7ad8b/0xf62499ab9277ea16 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775b0 expref: 496 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286211:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e1d7ad8b/0xf62499ab9277ea16 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775b0 expref: 496 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286295:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e1d7ad8b/0xf62499ab9277ea16 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775b0 expref: 496 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286307:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7172:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.286310:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7172:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.286315:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286321:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286342:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.286363:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+495:2407:x1871938519898240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 232us (665us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.286369:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99931 00000100:00100000:2.0:1785220581.286392:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510580352 00000100:00100000:2.0:1785220581.286395:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510580352 00000100:00100000:2.0:1785220581.286418:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284007 00000100:00100000:2.0:1785220581.286425:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1038:3655:x1871938510580352:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.286438:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29107 count 1 00010000:00010000:2.0:1785220581.286450:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286462:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab9c14aa/0xf62499ab92782fa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7933f expref: 441 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286469:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b3:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.286473:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.286477:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286497:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ab9c14aa/0xf62499ab92782fa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7933f expref: 441 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286508:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ab9c14aa/0xf62499ab92782fa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7933f expref: 441 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.286519:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510581760:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.286542:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.286596:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286605:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286614:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286621:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.286635:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.288210:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+961:3655:x1871938510578944:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11543us (11724us total) trans 4295069812 rc 0/0 00000004:00080000:1.0:1785220581.288256:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a6680/2, rc 0, transno 4295069812 00000100:00100000:1.0:1785220581.288271:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510578944:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.288325:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.288338:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510581888:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.288381:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.288449:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283997 00000100:00100000:2.0:1785220581.288473:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510581888 00000100:00100000:2.0:1785220581.288478:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510581888 00000100:00100000:2.0:1785220581.288521:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284008 00000100:00100000:2.0:1785220581.288528:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+961:3655:x1871938510581888:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.288550:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29049 count 1 00010000:00010000:2.0:1785220581.288572:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288593:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008f2188a1/0xf62499ab9277f808 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7789d expref: 494 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288602:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7179:0x0].0x0 since=35s v=(0 0) 00010000:00010000:2.0:1785220581.288606:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7179:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.288611:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288638:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008f2188a1/0xf62499ab9277f808 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7789d expref: 494 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288650:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008f2188a1/0xf62499ab9277f808 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7789d expref: 494 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.288663:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510582016:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.288721:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9600 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.288739:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.288747:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288755:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.288759:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288778:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288795:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288803:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4478 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7172:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.288864:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.288866:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.288868:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.288891:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.289168:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+441:2407:x1871938519897600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 8744us (11581us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.289178:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99925 00000004:00080000:2.0:1785220581.289353:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365f800/2, rc 0, transno 4295070408 00000100:00100000:2.0:1785220581.289366:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510577024:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.289417:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93609500/2, rc 0, transno 4295070409 00000100:00100000:2.0:1785220581.289425:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510573312:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.289467:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8aca80/2, rc 0, transno 4295069810 00000100:00100000:2.0:1785220581.289476:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510573824:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.289484:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.289490:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510582144:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.289523:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.289542:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510582144 00000100:00100000:3.0:1785220581.289548:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510582144 00000100:00100000:3.0:1785220581.289580:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284009 00000100:00100000:3.0:1785220581.289589:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b033480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1039:3654:x1871938510582144:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.289607:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29110 count 1 00010000:00010000:3.0:1785220581.289625:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289639:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000de23c7e5/0xf62499ab92783293 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7948f expref: 440 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289646:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b6:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.289649:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.289653:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289673:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000de23c7e5/0xf62499ab92783293 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7948f expref: 440 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289682:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000de23c7e5/0xf62499ab92783293 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7948f expref: 440 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.289693:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510582400:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.289722:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9adf2a80 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.289755:0:14144:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e5bd3942/0xf62499ab9277f5a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77746 expref: 494 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.289779:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510579968:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.289801:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289811:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289818:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289828:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289842:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.289861:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.290005:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+962:3655:x1871938510579456:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13343us (13447us total) trans 4295069813 rc 0/0 00000100:00100000:3.0:1785220581.290013:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283998 00010000:00010000:3.0:1785220581.290033:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.290038:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.290339:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510580480:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.290357:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.290391:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.290427:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.290443:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519898368 00000100:00100000:0.0:1785220581.290447:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519898368 00010000:00010000:0.0:1785220581.290467:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000084f15ec0/0xf62499ab92782ecd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792dd expref: 441 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.290490:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519898496 00000100:00100000:0.0:1785220581.290493:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519898496 00010000:00010000:0.0:1785220581.290506:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000075d68210/0xf62499ab9277f481 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b776cf expref: 494 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.290527:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99932 00000100:00100000:0.0:1785220581.290534:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+441:2406:x1871938519898368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.290545:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.290549:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000084f15ec0/0xf62499ab92782ecd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792dd expref: 441 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290559:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000084f15ec0/0xf62499ab92782ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792dd expref: 441 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290578:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000084f15ec0/0xf62499ab92782ecd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792dd expref: 441 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290589:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.290592:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.290597:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290605:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290627:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.290650:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+440:2406:x1871938519898368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.290658:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99932 00000100:00100000:0.0:1785220581.290671:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99933 00000100:00100000:0.0:1785220581.290678:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+494:2407:x1871938519898496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.290687:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.290692:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000075d68210/0xf62499ab9277f481 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b776cf expref: 494 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290703:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000075d68210/0xf62499ab9277f481 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b776cf expref: 494 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290722:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7175:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.290725:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7175:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.290729:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290738:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290751:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.290783:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+494:2407:x1871938519898496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.290790:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99933 00010000:00010000:0.0:1785220581.290808:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290817:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290825:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290833:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.290846:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.290959:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1040:3652:x1871938510580224:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11489us (11622us total) trans 4295070412 rc 0/0 00000100:00100000:0.0:1785220581.290966:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284001 00010000:00010000:0.0:1785220581.290986:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.290992:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.291001:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.291006:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.291015:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.291029:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.291036:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4486 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.291055:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.291058:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.291059:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.291540:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.291544:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291554:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291566:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291574:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4471 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7176:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291596:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.291599:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.291601:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.291614:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291624:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291633:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291641:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291659:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.291758:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1040:3653:x1871938510578176:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22601us (25810us total) trans 4295070413 rc 0/0 00000100:00100000:3.0:1785220581.291779:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283994 00010000:00010000:3.0:1785220581.291802:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.291807:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291813:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.291816:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291823:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291893:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291901:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a445c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291917:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.291918:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.291920:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.291932:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291939:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291946:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291952:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.291966:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.292039:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1040:3653:x1871938510578560:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21457us (22240us total) trans 4295070414 rc 0/0 00000100:00100000:3.0:1785220581.292047:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283995 00010000:00010000:3.0:1785220581.292065:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.292070:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.292077:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.292080:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.292416:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510580864:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.292438:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.292481:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.292522:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510581120:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.292536:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.292578:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510581248:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.292579:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.292591:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.292592:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.292600:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4463 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.292625:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.292627:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.292628:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.292718:0:14197:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075d68210/0xf62499ab9277f481 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b776cf expref: 493 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.292731:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510580608:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.292745:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.292753:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.292759:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.292921:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.292934:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.292954:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.293070:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+962:3655:x1871938510579328:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13606us (16032us total) trans 4295069814 rc 0/0 00000100:00100000:1.0:1785220581.293079:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284000 00010000:00010000:0.0:1785220581.293171:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.293177:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293185:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.293189:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293197:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293209:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.293219:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519898624 00000100:00100000:1.0:1785220581.293225:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519898624 00010000:00010000:1.0:1785220581.293249:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006d3bdb01/0xf62499ab9277eb89 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775fd expref: 493 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.293277:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99934 00000100:00100000:1.0:1785220581.293285:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+493:2406:x1871938519898624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.293300:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.293306:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006d3bdb01/0xf62499ab9277eb89 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775fd expref: 493 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.293316:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006d3bdb01/0xf62499ab9277eb89 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775fd expref: 493 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293333:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a448d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7175:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.293334:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d3bdb01/0xf62499ab9277eb89 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b775fd expref: 493 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293351:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.293352:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7173:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.293353:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.293355:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.293356:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7173:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.293361:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.293370:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.293381:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.293390:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.293397:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519898752 00000100:00100000:0.0:1785220581.293400:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519898752 00010000:00010000:0.0:1785220581.293415:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a1c71f47/0xf62499ab9277f664 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b777af expref: 493 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.293419:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+493:2406:x1871938519898624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 136us (939us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.293426:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99934 00000100:00100000:0.0:1785220581.293436:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99935 00000100:00100000:0.0:1785220581.293442:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+492:2404:x1871938519898752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.293450:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.293454:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a1c71f47/0xf62499ab9277f664 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b777af expref: 492 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293463:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a1c71f47/0xf62499ab9277f664 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b777af expref: 492 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293475:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a1c71f47/0xf62499ab9277f664 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b777af expref: 492 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293485:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7177:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.293488:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7177:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.293492:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293504:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.293511:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510582528:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.293521:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.293539:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+491:2404:x1871938519898752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.293546:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99935 00000100:00100000:1.0:1785220581.293547:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.293564:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293572:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293580:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.293585:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbdc00/2, rc 0, transno 4295070414 00010000:00010000:0.0:1785220581.293587:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293598:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.293601:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510578560:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.293612:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbf480/2, rc 0, transno 4295070413 00000100:00100000:1.0:1785220581.293616:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510578176:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.293628:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.293634:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510582272:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.293657:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.293681:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+962:3653:x1871938510579072:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10234us (10289us total) trans 4295069815 rc 0/0 00000004:00080000:1.0:1785220581.293682:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a7480/2, rc 0, transno 4295069815 00000100:00100000:0.0:1785220581.293688:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284002 00000100:00100000:1.0:1785220581.293689:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510579072:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.293700:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510582528 00000100:00100000:0.0:1785220581.293703:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510582528 00000100:00100000:0.0:1785220581.293722:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510582272 00000100:00100000:0.0:1785220581.293725:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510582272 00000004:00080000:1.0:1785220581.293741:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a4a80/2, rc 0, transno 4295069814 00000100:00100000:0.0:1785220581.293741:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284010 00000100:00100000:0.0:1785220581.293747:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+962:3653:x1871938510582528:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.293748:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510579328:0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.293780:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29050 count 1 00010000:00010000:0.0:1785220581.293798:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.293811:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000195f91f/0xf62499ab9277f8c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77906 expref: 490 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.293814:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a5180/2, rc 0, transno 4295069813 00010000:00010000:0.0:1785220581.293819:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x717a:0x0].0x0 since=35s v=(0 0) 00000100:00100000:1.0:1785220581.293821:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510579456:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.293822:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.293826:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.293848:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.293854:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510582912:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.293875:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.293897:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510582784:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.293917:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.293931:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.293935:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510582656:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.293955:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.293973:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510582912 00000100:00100000:2.0:1785220581.293977:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510582912 00000004:00080000:1.0:1785220581.293985:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93664380/2, rc 0, transno 4295070412 00000100:00100000:1.0:1785220581.293990:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510580224:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.294001:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510582784 00000100:00100000:2.0:1785220581.294003:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510582784 00000100:00100000:2.0:1785220581.294018:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510582656 00000100:00100000:2.0:1785220581.294021:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510582656 00000100:00100000:2.0:1785220581.294036:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284011 00000100:00100000:2.0:1785220581.294042:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1040:3653:x1871938510582272:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.294054:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29111 count 1 00000100:00100000:1.0:1785220581.294059:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.294068:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.294080:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cf22b76a/0xf62499ab92783350 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794ea expref: 439 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.294270:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284012 00000100:00100000:3.0:1785220581.294281:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+965:3653:x1871938510582912:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.294293:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29053 count 1 00010000:00010000:3.0:1785220581.294305:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294316:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009a5e2d03/0xf62499ab9277fb65 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a10 expref: 490 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294324:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x717d:0x0].0x0 since=35s v=(0 0) 00010000:00010000:3.0:1785220581.294327:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.294330:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294354:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009a5e2d03/0xf62499ab9277fb65 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a10 expref: 490 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.294379:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000195f91f/0xf62499ab9277f8c5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77906 expref: 490 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294389:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009a5e2d03/0xf62499ab9277fb65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a10 expref: 490 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.294389:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000195f91f/0xf62499ab9277f8c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77906 expref: 490 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.294403:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510583040:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.294403:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510583168:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.294451:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b7:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.294454:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.294458:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.294482:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cf22b76a/0xf62499ab92783350 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794ea expref: 439 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.294496:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cf22b76a/0xf62499ab92783350 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794ea expref: 439 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.294517:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510583296:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.294568:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a76d180 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.294568:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.294582:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.294584:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284013 00000100:00100000:3.0:1785220581.294592:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+965:3655:x1871938510582784:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.294596:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.294603:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29052 count 1 00010000:00010000:0.0:1785220581.294605:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294614:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.294614:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294625:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001917bf72/0xf62499ab9277f9dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779ae expref: 490 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.294628:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.294632:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x717c:0x0].0x0 since=35s v=(0 0) 00010000:00010000:3.0:1785220581.294635:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.294639:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294657:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001917bf72/0xf62499ab9277f9dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779ae expref: 490 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294666:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001917bf72/0xf62499ab9277f9dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779ae expref: 490 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.294676:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510583424:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.294697:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec206e6c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.294714:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.294720:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294727:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.294731:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.294732:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+966:3653:x1871938510579584:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10912us (11270us total) trans 4295069816 rc 0/0 00000100:00100000:0.0:1785220581.294740:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284003 00010000:00010000:3.0:1785220581.294741:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294754:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.294757:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284014 00010000:00010000:3.0:1785220581.294763:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4494 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7173:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.294763:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a4a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+965:3655:x1871938510582656:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.294787:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29051 count 1 00010000:00010000:0.0:1785220581.294797:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294804:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.294808:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.294808:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c81ed1a4/0xf62499ab9277f958 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7795a expref: 490 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294816:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.294816:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x717b:0x0].0x0 since=35s v=(0 0) 00010000:00010000:3.0:1785220581.294819:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.294819:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.294823:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294827:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294838:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294845:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a449b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7177:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.294860:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.294862:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.294864:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.294871:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.294918:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c81ed1a4/0xf62499ab9277f958 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7795a expref: 490 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.294929:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c81ed1a4/0xf62499ab9277f958 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7795a expref: 490 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.294930:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220581.294938:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510583808:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.294939:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a9b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510583552:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.294959:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.294970:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.295011:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510583552 00000100:00100000:0.0:1785220581.295017:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510583552 00000100:00100000:0.0:1785220581.295055:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284015 00000100:00100000:0.0:1785220581.295061:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1041:3652:x1871938510583552:12345-0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.295067:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97987b80/2, rc 0, transno 4295069816 00002000:00080000:0.0:1785220581.295073:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29112 count 1 00000100:00100000:1.0:1785220581.295075:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510579584:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.295145:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295157:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000b9cd1dd/0xf62499ab9278347d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7957d expref: 439 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295164:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b8:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.295167:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.295170:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295195:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000b9cd1dd/0xf62499ab9278347d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7957d expref: 439 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295205:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000b9cd1dd/0xf62499ab9278347d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7957d expref: 439 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.295291:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510584064:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.295319:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.295340:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510581504:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.295346:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519898880 00000100:00100000:0.0:1785220581.295349:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519898880 00010000:00010000:1.0:1785220581.295357:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295367:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005db1b4c9/0xf62499ab9278308d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793bd expref: 440 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.295397:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.295398:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99936 00000100:00100000:1.0:1785220581.295404:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510584192:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.295405:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+440:2406:x1871938519898880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.295413:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.295418:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005db1b4c9/0xf62499ab9278308d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793bd expref: 440 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.295427:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.295429:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005db1b4c9/0xf62499ab9278308d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793bd expref: 440 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295444:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005db1b4c9/0xf62499ab9278308d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793bd expref: 440 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.295448:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510583936:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.295459:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.295463:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b4:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.295467:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.295468:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295476:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.295482:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.295486:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97840000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510583680:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.295494:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.295503:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.295507:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+439:2406:x1871938519898880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (1448us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.295514:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99936 00010000:00010000:0.0:1785220581.295530:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295538:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295547:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295554:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295565:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.295578:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510581632:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.295590:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.295669:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1042:3653:x1871938510580992:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11659us (12032us total) trans 4295070415 rc 0/0 00000004:00080000:1.0:1785220581.295674:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbd880/2, rc 0, transno 4295070415 00000100:00100000:0.0:1785220581.295676:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284004 00000100:00100000:1.0:1785220581.295680:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510580992:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.295688:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510584192 00000100:00100000:0.0:1785220581.295691:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510584192 00000100:00100000:0.0:1785220581.295707:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510583936 00000100:00100000:0.0:1785220581.295710:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510583936 00000100:00100000:0.0:1785220581.295723:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510583680 00000100:00100000:0.0:1785220581.295725:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510583680 00000100:00100000:1.0:1785220581.295734:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.295739:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284016 00000100:00100000:0.0:1785220581.295744:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+966:3653:x1871938510584192:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.295756:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29054 count 1 00000100:00100000:1.0:1785220581.295763:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.295780:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295792:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007b95297c/0xf62499ab9277fc30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a80 expref: 490 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295798:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x717e:0x0].0x0 since=35s v=(0 0) 00010000:00010000:0.0:1785220581.295801:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.295805:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.295806:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899136 00000100:00100000:3.0:1785220581.295810:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899136 00010000:00010000:0.0:1785220581.295824:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007b95297c/0xf62499ab9277fc30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a80 expref: 491 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.295826:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000540330bf/0xf62499ab9277f760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7782d expref: 491 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.295834:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007b95297c/0xf62499ab9277fc30 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a80 expref: 491 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.295847:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99937 00000100:00100000:3.0:1785220581.295854:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+491:2407:x1871938519899136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.295879:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.295885:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000540330bf/0xf62499ab9277f760 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7782d expref: 491 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.295895:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000540330bf/0xf62499ab9277f760 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7782d expref: 491 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.295909:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000540330bf/0xf62499ab9277f760 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7782d expref: 491 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.295920:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7178:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.295923:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7178:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.295926:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.295933:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.295957:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.295974:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+490:2407:x1871938519899136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 122us (241us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.295981:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99937 00000100:00100000:3.0:1785220581.296004:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284017 00000100:00100000:3.0:1785220581.296011:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1043:3655:x1871938510583936:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.296017:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510584320:192.168.201.51@tcp:104: 00002000:00080000:3.0:1785220581.296023:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29114 count 1 00010000:00010000:3.0:1785220581.296035:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.296041:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.296046:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000790b367b/0xf62499ab9278371d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79687 expref: 438 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296053:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ba:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.296056:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.296059:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296078:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000790b367b/0xf62499ab9278371d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79687 expref: 438 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296108:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000790b367b/0xf62499ab9278371d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79687 expref: 438 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.296118:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510584448:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.296225:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920c00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.296243:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284018 00000100:00100000:3.0:1785220581.296252:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1043:3655:x1871938510583680:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.296263:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29113 count 1 00010000:00010000:3.0:1785220581.296274:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296284:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000094654270/0xf62499ab927835d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79610 expref: 438 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296291:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71b9:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.296294:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.296297:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296464:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000094654270/0xf62499ab927835d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79610 expref: 438 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296475:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000094654270/0xf62499ab927835d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79610 expref: 438 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.296485:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510584576:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.296508:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.296524:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.296530:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296538:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.296542:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296680:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296693:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296701:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a44a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296719:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.296721:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.296722:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.296787:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220581.296797:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97840e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510584704:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.296826:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.296848:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510584704 00000100:00100000:3.0:1785220581.296853:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510584704 00000100:00100000:3.0:1785220581.296882:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284019 00000100:00100000:3.0:1785220581.296889:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97843b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1044:3652:x1871938510584704:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.296901:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29115 count 1 00010000:00010000:3.0:1785220581.296914:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296926:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000052927ad6/0xf62499ab92783804 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796f0 expref: 438 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296937:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71bb:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.296939:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.296943:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296960:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000052927ad6/0xf62499ab92783804 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796f0 expref: 438 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.296970:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000052927ad6/0xf62499ab92783804 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796f0 expref: 438 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.296980:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510584832:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.296981:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.297000:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.297020:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510581760:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.297037:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.297071:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510582016:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.297082:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297475:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297485:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297494:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297501:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297518:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.297655:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510580096:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.297662:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+967:3655:x1871938510580736:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11757us (11993us total) trans 4295069817 rc 0/0 00010000:00010000:3.0:1785220581.297668:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.297669:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284006 00010000:00010000:0.0:1785220581.297690:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.297696:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297703:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.297706:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297714:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297727:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297734:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a44b0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7178:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.297749:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.297751:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.297753:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.298277:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbce00/2, rc 0, transno 4295069817 00000100:00100000:2.0:1785220581.298314:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899392 00000100:00100000:2.0:1785220581.298320:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899392 00000100:00100000:1.0:1785220581.298322:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510580736:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.298343:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008f2188a1/0xf62499ab9277f808 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7789d expref: 490 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.298360:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.298366:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899008 00000100:00100000:2.0:1785220581.298368:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899008 00000100:00100000:1.0:1785220581.298378:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.298381:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d2af55e1/0xf62499ab927831b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7942d expref: 439 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.298401:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899264 00000100:00100000:2.0:1785220581.298404:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899264 00000100:00100000:1.0:1785220581.298408:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.298414:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ab9c14aa/0xf62499ab92782fa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7933f expref: 440 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.298447:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899520 00000100:00100000:2.0:1785220581.298451:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899520 00010000:00010000:2.0:1785220581.298463:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e5b9f022/0xf62499ab9277e825 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77524 expref: 491 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.298481:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99938 00000100:00100000:2.0:1785220581.298489:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+491:2407:x1871938519899392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.298501:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.298505:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008f2188a1/0xf62499ab9277f808 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7789d expref: 491 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298515:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008f2188a1/0xf62499ab9277f808 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7789d expref: 491 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298531:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008f2188a1/0xf62499ab9277f808 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7789d expref: 491 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298543:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7179:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298547:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7179:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298552:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298560:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298585:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.298617:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+490:2407:x1871938519899392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 129us (2852us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.298624:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99938 00000100:00100000:2.0:1785220581.298665:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99939 00000100:00100000:2.0:1785220581.298672:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+440:2407:x1871938519899008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.298680:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.298685:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d2af55e1/0xf62499ab927831b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7942d expref: 440 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298696:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d2af55e1/0xf62499ab927831b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7942d expref: 440 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298709:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d2af55e1/0xf62499ab927831b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7942d expref: 440 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298717:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298720:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298724:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298731:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298752:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.298782:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+439:2407:x1871938519899008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (421us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.298789:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99939 00000100:00100000:2.0:1785220581.298803:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99940 00000100:00100000:2.0:1785220581.298810:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+438:2407:x1871938519899264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.298818:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.298823:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ab9c14aa/0xf62499ab92782fa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7933f expref: 438 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298832:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ab9c14aa/0xf62499ab92782fa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7933f expref: 438 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298844:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab9c14aa/0xf62499ab92782fa6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7933f expref: 438 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298854:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298857:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298861:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298868:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298890:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.298908:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+437:2407:x1871938519899264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.298915:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99940 00000100:00100000:2.0:1785220581.298929:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99941 00000100:00100000:2.0:1785220581.298936:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6eca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+489:2407:x1871938519899520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.298943:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.298947:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e5b9f022/0xf62499ab9277e825 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77524 expref: 489 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298957:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e5b9f022/0xf62499ab9277e825 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77524 expref: 489 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.298959:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.298968:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298970:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e5b9f022/0xf62499ab9277e825 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77524 expref: 489 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.298978:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298979:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7171:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298982:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7171:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.298986:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.298988:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.298994:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299004:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.299014:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.299031:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6eca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+488:2407:x1871938519899520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.299037:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99941 00010000:00010000:2.0:1785220581.299057:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299064:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299072:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299078:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299112:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.299455:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1046:3655:x1871938510580352:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13029us (13382us total) trans 4295070417 rc 0/0 00000100:00100000:2.0:1785220581.299463:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284007 00000100:00100000:1.0:1785220581.299473:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1045:3653:x1871938510581376:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13734us (13862us total) trans 4295070416 rc 0/0 00000100:00100000:1.0:1785220581.299480:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284005 00010000:00010000:2.0:1785220581.299484:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299493:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299505:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299512:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299513:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.299518:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.299525:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220581.299526:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.299530:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299538:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299550:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299557:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a44a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299574:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.299578:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299585:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.299588:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299595:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299605:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299611:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a44b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299623:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.299626:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299632:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.299635:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299642:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299656:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299663:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a44be lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7179:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.299676:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.299678:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.299680:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.300196:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.300208:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.300221:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.300228:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.300245:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.300344:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+968:3655:x1871938510578816:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22368us (23376us total) trans 4295069819 rc 0/0 00000100:00100000:3.0:1785220581.300351:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283999 00000004:00080000:0.0:1785220581.300378:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93609f80/2, rc 0, transno 4295070416 00000100:00100000:0.0:1785220581.300401:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510581376:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.300419:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.300423:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.300431:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fee00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.300435:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.300443:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.300461:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.300467:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000976ffad2/0xf62499ab927a447f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7171:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.300484:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.300485:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.300487:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.300539:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+967:3655:x1871938510581888:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12010us (12156us total) trans 4295069818 rc 0/0 00000004:00080000:1.0:1785220581.300539:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365dc00/2, rc 0, transno 4295069818 00000100:00100000:2.0:1785220581.300546:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284008 00010000:00010000:2.0:1785220581.300565:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.300571:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.300582:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510581888:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.300596:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93664e00/2, rc 0, transno 4295070417 00000100:00100000:1.0:1785220581.300610:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510580352:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.300621:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a7100/2, rc 0, transno 4295069819 00000100:00100000:0.0:1785220581.300679:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.300749:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510585216:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.300798:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.300839:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510585344:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.300861:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.300935:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510585216 00000100:00100000:2.0:1785220581.300941:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510585216 00000100:00100000:2.0:1785220581.300979:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510585344 00000100:00100000:2.0:1785220581.300982:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510585344 00000100:00100000:2.0:1785220581.301010:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284020 00000100:00100000:2.0:1785220581.301018:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1045:3653:x1871938510585216:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.301046:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29117 count 1 00000100:00100000:3.0:1785220581.301067:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284021 00010000:00010000:2.0:1785220581.301067:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.301076:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+967:3653:x1871938510585344:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.301110:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29056 count 1 00010000:00010000:3.0:1785220581.301124:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.301147:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007e76b5d9/0xf62499ab9277fe98 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b98 expref: 487 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.301156:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7180:0x0].0x0 since=35s v=(0 0) 00010000:00010000:3.0:1785220581.301160:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7180:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.301164:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.301190:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007e76b5d9/0xf62499ab9277fe98 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b98 expref: 487 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.301204:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007e76b5d9/0xf62499ab9277fe98 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b98 expref: 487 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.301218:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510585600:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.301272:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.301404:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510578816:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.301417:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510585088:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.301445:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.301473:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1785220581.301479:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510584960:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.301501:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.301516:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510585472:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.301536:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.301807:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510585088 00000100:00100000:0.0:1785220581.301813:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510585088 00000100:00100000:0.0:1785220581.301847:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510584960 00000100:00100000:0.0:1785220581.301851:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510584960 00000100:00100000:0.0:1785220581.301869:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510585472 00000100:00100000:0.0:1785220581.301872:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510585472 00000100:00100000:0.0:1785220581.301892:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284022 00000100:00100000:0.0:1785220581.301898:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1046:3655:x1871938510585088:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.301914:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29116 count 1 00010000:00010000:0.0:1785220581.301930:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.301949:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000638ebacd/0xf62499ab9278390e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7976e expref: 436 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.301956:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71bc:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.301959:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.301963:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.301969:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284023 00000100:00100000:1.0:1785220581.301978:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+969:3655:x1871938510584960:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.301985:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000638ebacd/0xf62499ab9278390e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7976e expref: 436 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220581.301989:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29055 count 1 00010000:00010000:0.0:1785220581.301995:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000638ebacd/0xf62499ab9278390e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7976e expref: 436 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.302002:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.302006:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510585728:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.302013:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cdc6ba65/0xf62499ab9277fd02 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ae9 expref: 487 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.302019:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x717f:0x0].0x0 since=35s v=(0 0) 00010000:00010000:1.0:1785220581.302022:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.302025:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.302037:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9ae74c00 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.302042:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cdc6ba65/0xf62499ab9277fd02 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ae9 expref: 487 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.302051:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cdc6ba65/0xf62499ab9277fd02 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ae9 expref: 487 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.302054:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284024 00000100:00100000:0.0:1785220581.302060:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+969:3655:x1871938510585472:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.302061:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510585856:192.168.201.51@tcp:104: 00002000:00080000:0.0:1785220581.302071:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29057 count 1 00000100:00100000:1.0:1785220581.302083:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01b40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.302083:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.302112:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001250a3df/0xf62499ab9277ffa9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c24 expref: 487 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.302124:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7181:0x0].0x0 since=35s v=(0 0) 00010000:00010000:0.0:1785220581.302127:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7181:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.302130:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.302149:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001250a3df/0xf62499ab9277ffa9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c24 expref: 487 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.302159:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001250a3df/0xf62499ab9277ffa9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c24 expref: 487 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.302167:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510585984:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.302193:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e82bfd480 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.302338:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000036845922/0xf62499ab927839d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b797d7 expref: 436 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.302349:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71bd:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.302353:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.302357:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.302380:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000036845922/0xf62499ab927839d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b797d7 expref: 436 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.302389:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000036845922/0xf62499ab927839d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b797d7 expref: 436 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.302399:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:13837:1871938510586112:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.302423:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.307750:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.307750:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510583424:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.307794:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.307827:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510583808:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.307844:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.307941:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510584064:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.307942:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.307958:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.307965:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899648 00000100:00100000:1.0:1785220581.307971:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899648 00000100:00100000:0.0:1785220581.307979:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510584320:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.307992:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308004:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009a5e2d03/0xf62499ab9277fb65 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a10 expref: 488 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.308011:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510583168:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.308021:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.308035:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899904 00000100:00100000:1.0:1785220581.308038:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899904 00000100:00100000:0.0:1785220581.308040:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99942 00000100:00100000:0.0:1785220581.308048:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+489:2407:x1871938519899648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.308050:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000195f91f/0xf62499ab9277f8c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77906 expref: 489 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.308061:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220581.308067:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99943 00010000:00010000:0.0:1785220581.308068:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009a5e2d03/0xf62499ab9277fb65 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a10 expref: 489 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.308073:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+489:2404:x1871938519899904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.308079:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.308079:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009a5e2d03/0xf62499ab9277fb65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a10 expref: 489 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308123:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000195f91f/0xf62499ab9277f8c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77906 expref: 489 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308135:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000195f91f/0xf62499ab9277f8c5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77906 expref: 489 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308166:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000195f91f/0xf62499ab9277f8c5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77906 expref: 489 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308180:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.308184:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.308189:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308197:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308226:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.308271:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+488:2404:x1871938519899904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 197us (329us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.308279:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99943 00010000:00010000:1.0:1785220581.308304:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308313:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308324:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308332:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308347:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.308372:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510584448:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.308389:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.308456:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.308459:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.308463:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.308475:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.308486:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.308503:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+487:2407:x1871938519899648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 456us (762us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.308510:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99942 00000100:00100000:1.0:1785220581.308522:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+970:3653:x1871938510582528:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14774us (14973us total) trans 4295069820 rc 0/0 00000100:00100000:1.0:1785220581.308529:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284010 00000004:00080000:0.0:1785220581.308547:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365f480/2, rc 0, transno 4295069820 00010000:00010000:1.0:1785220581.308549:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.308556:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308564:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.308569:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308578:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308590:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.308603:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a44cc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.308604:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510582528:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.308622:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.308624:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.308625:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.308639:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.308731:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900032 00000100:00100000:2.0:1785220581.308735:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900032 00010000:00010000:2.0:1785220581.308757:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c81ed1a4/0xf62499ab9277f958 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7795a expref: 487 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.308832:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99944 00000100:00100000:2.0:1785220581.308841:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+487:2407:x1871938519900032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.308850:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.308855:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c81ed1a4/0xf62499ab9277f958 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7795a expref: 487 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.308866:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c81ed1a4/0xf62499ab9277f958 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7795a expref: 487 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.308880:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c81ed1a4/0xf62499ab9277f958 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7795a expref: 487 pid: 14193 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.308890:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.308893:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.308896:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.308903:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.308920:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.308941:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+486:2407:x1871938519900032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.308947:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99944 00010000:00010000:0.0:1785220581.308984:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.308994:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.309002:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.309010:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.309033:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.309059:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.309064:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.309072:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.309076:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.309083:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.309120:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.309126:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a44e8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.309195:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.309197:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.309199:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.309552:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510586240:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.309552:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a4a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+970:3655:x1871938510582656:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14788us (15594us total) trans 4295069821 rc 0/0 00000100:00100000:0.0:1785220581.309559:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284014 00000100:00100000:1.0:1785220581.309595:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.309635:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbc380/2, rc 0, transno 4295069821 00000100:00100000:1.0:1785220581.309647:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510582656:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.309689:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510586240 00000100:00100000:2.0:1785220581.309694:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510586240 00000100:00100000:2.0:1785220581.309721:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284025 00000100:00100000:2.0:1785220581.309729:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+970:3655:x1871938510586240:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.309751:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29058 count 1 00010000:00010000:2.0:1785220581.309781:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.309800:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000043a32727/0xf62499ab927800ba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c94 expref: 485 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.309807:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7182:0x0].0x0 since=35s v=(0 0) 00010000:00010000:2.0:1785220581.309810:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7182:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.309814:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.309837:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000043a32727/0xf62499ab927800ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c94 expref: 485 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.309850:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000043a32727/0xf62499ab927800ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c94 expref: 485 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.309861:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8441:1871938510586496:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.309909:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.310570:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.310584:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510586368:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.310611:0:14166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009a5e2d03/0xf62499ab9277fb65 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a10 expref: 485 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.310619:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.310625:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510583040:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.310645:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.310654:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.310660:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.310669:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.310677:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.310689:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.310697:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.310718:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.310732:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.310748:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.310778:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519899776 00000100:00100000:0.0:1785220581.310783:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519899776 00010000:00010000:0.0:1785220581.310803:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001917bf72/0xf62499ab9277f9dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779ae expref: 485 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.310825:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900416 00000100:00100000:0.0:1785220581.310828:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900416 00000100:00100000:3.0:1785220581.310829:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+971:3653:x1871938510582912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16551us (16952us total) trans 4295069822 rc 0/0 00010000:00010000:0.0:1785220581.310839:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007b95297c/0xf62499ab9277fc30 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a80 expref: 486 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.310857:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900288 00000100:00100000:0.0:1785220581.310860:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900288 00010000:00010000:0.0:1785220581.310872:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000790b367b/0xf62499ab9278371d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79687 expref: 437 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.310890:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900160 00000100:00100000:0.0:1785220581.310892:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900160 00000100:00100000:3.0:1785220581.310896:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284012 00010000:00010000:0.0:1785220581.310904:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000b9cd1dd/0xf62499ab9278347d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7957d expref: 438 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.310913:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510586368 00000100:00100000:3.0:1785220581.310915:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510586368 00000100:00100000:0.0:1785220581.310928:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99945 00000100:00100000:3.0:1785220581.310935:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284026 00000100:00100000:0.0:1785220581.310935:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+486:2407:x1871938519899776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.310940:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+971:3653:x1871938510586368:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.310944:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.310949:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001917bf72/0xf62499ab9277f9dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779ae expref: 486 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.310952:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29059 count 1 00010000:00010000:0.0:1785220581.310958:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001917bf72/0xf62499ab9277f9dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779ae expref: 486 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.310966:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.310974:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001917bf72/0xf62499ab9277f9dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b779ae expref: 486 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.310977:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ad6dbc9f/0xf62499ab927801d2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d27 expref: 486 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.310984:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7183:0x0].0x0 since=35s v=(0 0) 00010000:00010000:0.0:1785220581.310985:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.310988:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.310989:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7183:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.310993:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.310993:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.311001:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311016:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ad6dbc9f/0xf62499ab927801d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d27 expref: 485 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.311017:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.311027:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ad6dbc9f/0xf62499ab927801d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d27 expref: 485 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.311037:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+485:2407:x1871938519899776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.311043:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510586624:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.311043:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99945 00000100:00100000:3.0:1785220581.311071:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.311079:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99946 00010000:00010000:3.0:1785220581.311223:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.311229:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311237:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.311241:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311248:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311261:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311269:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a44da lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311288:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.311290:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.311291:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.311344:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99947 00000100:00100000:1.0:1785220581.311350:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99948 00000100:00100000:2.0:1785220581.311353:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+438:2404:x1871938519900288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.311357:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+438:2404:x1871938519900160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.311364:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.311364:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.311369:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000b9cd1dd/0xf62499ab9278347d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7957d expref: 438 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.311370:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000790b367b/0xf62499ab9278371d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79687 expref: 438 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.311381:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000790b367b/0xf62499ab9278371d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79687 expref: 438 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.311381:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000b9cd1dd/0xf62499ab9278347d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7957d expref: 438 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.311396:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000790b367b/0xf62499ab9278371d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79687 expref: 438 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.311396:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9cd1dd/0xf62499ab9278347d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7957d expref: 438 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.311408:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.311408:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.311412:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.311412:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.311417:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.311417:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.311427:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.311427:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.311446:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.311454:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.311466:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+436:2404:x1871938519900160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 110us (716us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.311467:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+436:2404:x1871938519900288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 117us (735us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.311473:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99948 00000100:00100000:0.0:1785220581.311501:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+484:2407:x1871938519900416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.311502:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.311510:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.311515:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007b95297c/0xf62499ab9277fc30 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a80 expref: 484 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311516:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311526:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.311526:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007b95297c/0xf62499ab9277fc30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a80 expref: 484 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311534:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.311539:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007b95297c/0xf62499ab9277fc30 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a80 expref: 484 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311548:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.311548:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.311551:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.311555:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.311561:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.311578:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.311595:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+483:2407:x1871938519900416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 511us (877us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.311627:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99947 00000100:00100000:3.0:1785220581.311665:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1047:3655:x1871938510583936:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15654us (16196us total) trans 4295070418 rc 0/0 00000100:00100000:3.0:1785220581.311672:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284017 00010000:00010000:3.0:1785220581.311687:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.311692:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311700:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.311703:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311712:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311723:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.311731:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a44fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.311737:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510583296:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.311745:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.311747:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.311749:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.311750:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.311777:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510584576:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.311791:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.311815:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510584832:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.311820:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.311827:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.311846:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.311917:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.311952:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.312018:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.312037:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.312054:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.312069:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900672 00000100:00100000:2.0:1785220581.312074:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900672 00010000:00010000:2.0:1785220581.312118:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cf22b76a/0xf62499ab92783350 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794ea expref: 435 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312137:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900800 00000100:00100000:2.0:1785220581.312140:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900800 00010000:00010000:2.0:1785220581.312152:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000052927ad6/0xf62499ab92783804 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796f0 expref: 436 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312168:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900544 00000100:00100000:2.0:1785220581.312171:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900544 00010000:00010000:2.0:1785220581.312182:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000094654270/0xf62499ab927835d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79610 expref: 437 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312197:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519900928 00000100:00100000:2.0:1785220581.312200:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519900928 00010000:00010000:2.0:1785220581.312212:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000de23c7e5/0xf62499ab92783293 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7948f expref: 438 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312261:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901056 00000100:00100000:1.0:1785220581.312265:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901056 00010000:00010000:1.0:1785220581.312281:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007e76b5d9/0xf62499ab9277fe98 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b98 expref: 484 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312295:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901312 00000100:00100000:1.0:1785220581.312297:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901312 00010000:00010000:1.0:1785220581.312309:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cdc6ba65/0xf62499ab9277fd02 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ae9 expref: 485 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312320:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901184 00000100:00100000:1.0:1785220581.312323:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901184 00010000:00010000:1.0:1785220581.312333:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001250a3df/0xf62499ab9277ffa9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c24 expref: 486 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312348:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99949 00000100:00100000:1.0:1785220581.312354:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+438:2404:x1871938519900672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.312362:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.312367:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cf22b76a/0xf62499ab92783350 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794ea expref: 438 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312375:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cf22b76a/0xf62499ab92783350 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794ea expref: 438 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312388:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf22b76a/0xf62499ab92783350 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794ea expref: 438 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312398:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.312401:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.312405:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312413:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312431:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.312457:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+437:2404:x1871938519900672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (636us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.312463:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99949 00000100:00100000:1.0:1785220581.312478:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99950 00000100:00100000:1.0:1785220581.312484:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+436:2406:x1871938519900800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.312513:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.312520:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000052927ad6/0xf62499ab92783804 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796f0 expref: 436 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312529:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000052927ad6/0xf62499ab92783804 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796f0 expref: 436 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312533:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99951 00000100:00100000:2.0:1785220581.312540:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+436:2406:x1871938519900544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.312542:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052927ad6/0xf62499ab92783804 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796f0 expref: 436 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312548:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.312552:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000094654270/0xf62499ab927835d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79610 expref: 435 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312553:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.312556:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.312560:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312568:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000094654270/0xf62499ab927835d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79610 expref: 435 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312569:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312581:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094654270/0xf62499ab927835d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79610 expref: 435 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312582:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.312590:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.312593:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b9:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.312597:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+434:2406:x1871938519900800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (751us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.312598:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312604:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99950 00010000:00010000:2.0:1785220581.312606:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312616:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99952 00010000:00010000:2.0:1785220581.312622:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.312622:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+486:2406:x1871938519901056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.312629:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.312634:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007e76b5d9/0xf62499ab9277fe98 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b98 expref: 486 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312638:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+433:2406:x1871938519900544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (722us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.312645:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99951 00010000:00010000:1.0:1785220581.312649:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007e76b5d9/0xf62499ab9277fe98 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b98 expref: 486 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312657:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99953 00010000:00010000:1.0:1785220581.312661:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7180:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220581.312663:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+486:2404:x1871938519901312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.312664:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7180:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.312669:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312670:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.312675:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cdc6ba65/0xf62499ab9277fd02 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ae9 expref: 486 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312677:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.312689:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.312690:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cdc6ba65/0xf62499ab9277fd02 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ae9 expref: 486 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312701:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717f:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.312704:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+486:2406:x1871938519901056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (686us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.312705:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x717f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.312709:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312710:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99952 00010000:00010000:2.0:1785220581.312716:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.312722:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99954 00010000:00010000:2.0:1785220581.312727:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.312728:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+485:2406:x1871938519901184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.312734:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.312739:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001250a3df/0xf62499ab9277ffa9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c24 expref: 485 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312741:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+485:2404:x1871938519901312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (704us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.312747:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99953 00010000:00010000:1.0:1785220581.312749:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001250a3df/0xf62499ab9277ffa9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c24 expref: 485 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.312758:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99955 00000100:00100000:2.0:1785220581.312775:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+432:2406:x1871938519900928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.312794:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.312799:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000de23c7e5/0xf62499ab92783293 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7948f expref: 432 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312808:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000de23c7e5/0xf62499ab92783293 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7948f expref: 432 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312824:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.312827:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.312831:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312838:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312849:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.312865:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+432:2406:x1871938519900928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (913us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.312871:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99955 00010000:00010000:2.0:1785220581.312895:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312904:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312912:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312919:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.312932:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.313054:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1047:3653:x1871938510582272:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19011us (19394us total) trans 4295070419 rc 0/0 00010000:00010000:3.0:1785220581.313247:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313257:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313265:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313273:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313290:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.313300:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284011 00000100:00100000:3.0:1785220581.313361:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1047:3655:x1871938510583680:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17112us (17857us total) trans 4295070420 rc 0/0 00000100:00100000:3.0:1785220581.313367:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284018 00000100:00100000:2.0:1785220581.313376:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:13837:1871938510586112:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.313383:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.313388:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313394:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313395:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.313399:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313407:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313420:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313427:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a44d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313439:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.313454:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901568 00000100:00100000:2.0:1785220581.313458:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901568 00010000:00010000:3.0:1785220581.313460:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.313463:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313474:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.313474:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000036845922/0xf62499ab927839d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b797d7 expref: 432 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313478:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313485:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313492:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99956 00010000:00010000:3.0:1785220581.313495:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313498:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+432:2407:x1871938519901568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.313503:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4504 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313506:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.313510:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000036845922/0xf62499ab927839d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b797d7 expref: 432 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313519:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.313519:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000036845922/0xf62499ab927839d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b797d7 expref: 432 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313521:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.313523:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.313532:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036845922/0xf62499ab927839d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b797d7 expref: 432 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313541:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.313543:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e76b5d9/0xf62499ab9277fe98 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77b98 expref: 484 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313544:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.313549:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.313555:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510585600:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.313556:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313566:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313570:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.313575:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313582:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313587:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+431:2407:x1871938519901568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (147us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.313590:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313593:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99956 00010000:00010000:3.0:1785220581.313598:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313609:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.313610:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313617:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313624:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313631:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313642:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.313690:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+972:3653:x1871938510585344:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12616us (12827us total) trans 4295069823 rc 0/0 00000100:00100000:3.0:1785220581.313697:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284021 00000100:00100000:2.0:1785220581.313702:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1047:3653:x1871938510585216:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12686us (12905us total) trans 4295070421 rc 0/0 00000100:00100000:2.0:1785220581.313707:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284020 00010000:00010000:3.0:1785220581.313713:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.313718:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313725:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.313728:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313736:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313746:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313754:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4519 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7180:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313757:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.313783:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1785220581.313786:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901440 00010000:00010000:3.0:1785220581.313787:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313789:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901440 00010000:00010000:3.0:1785220581.313793:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fec00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.313797:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313804:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000638ebacd/0xf62499ab9278390e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7976e expref: 431 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313805:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313815:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313823:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4512 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313823:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99957 00000100:00100000:2.0:1785220581.313829:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+431:2407:x1871938519901440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.313835:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.313839:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000638ebacd/0xf62499ab9278390e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7976e expref: 431 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313846:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.313848:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.313849:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000638ebacd/0xf62499ab9278390e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7976e expref: 431 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313850:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.313855:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.313861:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.313865:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.313869:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313874:0:14247:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de23c7e5/0xf62499ab92783293 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7948f expref: 431 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.313876:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.313886:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510582400:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.313888:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.313898:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313903:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+430:2407:x1871938519901440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (147us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.313907:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.313910:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99957 00010000:00010000:3.0:1785220581.313914:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313922:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313930:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.313941:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.314014:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b033480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1047:3654:x1871938510582144:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24426us (24490us total) trans 4295070422 rc 0/0 00000100:00100000:3.0:1785220581.314020:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284009 00010000:00010000:3.0:1785220581.314038:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.314041:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314052:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.314055:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314062:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314072:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314079:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a44c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314114:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.314116:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.314117:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.314125:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314133:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314140:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314151:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314168:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.314244:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97843b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1047:3652:x1871938510584704:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17355us (17415us total) trans 4295070423 rc 0/0 00000100:00100000:3.0:1785220581.314249:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284019 00010000:00010000:2.0:1785220581.314318:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.314323:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.314330:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.314332:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.314340:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.314351:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.314358:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a450b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.314373:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.314375:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.314377:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.314435:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99946 00010000:00010000:0.0:1785220581.314485:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.314498:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.314507:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.314514:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.314531:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.314617:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+972:3655:x1871938510582784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20026us (20699us total) trans 4295069824 rc 0/0 00000100:00100000:0.0:1785220581.314624:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284013 00010000:00010000:3.0:1785220581.314630:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.314635:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314642:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.314645:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314652:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314662:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314668:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a44e1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.314684:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.314686:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.314687:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.314716:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.314724:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.314732:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.314738:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.314743:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7181:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.314747:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7181:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.314753:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.314761:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.314788:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.314825:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+482:2406:x1871938519901184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2096us (2769us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.314832:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99954 00010000:00010000:1.0:1785220581.314866:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.314874:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.314881:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.314887:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.315927:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220581.315930:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.316059:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.316065:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316076:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.316079:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316109:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316125:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316132:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a44f6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316151:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.316155:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316161:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.316164:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316172:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316182:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316190:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a44ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.316208:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.316210:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.316211:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.316229:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.316438:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+972:3653:x1871938510584192:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20691us (21007us total) trans 4295069825 rc 0/0 00000100:00100000:0.0:1785220581.316447:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284016 00010000:00010000:0.0:1785220581.316502:0:14173:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000638ebacd/0xf62499ab9278390e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7976e expref: 429 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.316516:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510585728:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.316534:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316543:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316549:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316557:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316564:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316586:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.316672:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1048:3655:x1871938510585088:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14773us (15224us total) trans 4295070425 rc 0/0 00000100:00100000:0.0:1785220581.316679:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284022 00010000:00010000:0.0:1785220581.316711:0:14203:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001250a3df/0xf62499ab9277ffa9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c24 expref: 481 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.316724:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510585984:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.316734:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316742:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316748:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316756:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316762:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.316800:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.317188:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.317195:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317203:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.317207:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317215:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317230:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317238:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4520 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317256:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.317260:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317267:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.317270:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317282:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317292:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317299:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a452e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7181:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.317312:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.317313:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.317315:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.317589:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+972:3655:x1871938510585472:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15528us (16051us total) trans 4295069826 rc 0/0 00000100:00100000:0.0:1785220581.317597:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284024 00000004:00080000:0.0:1785220581.317649:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba5500/2, rc 0, transno 4295069823 00000100:00100000:0.0:1785220581.317666:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510585344:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.317681:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b031180/2, rc 0, transno 4295070421 00000100:00100000:0.0:1785220581.317694:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510585216:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.317707:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6eed80/2, rc 0, transno 4295069825 00000100:00100000:0.0:1785220581.317711:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510584192:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.317720:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ed500/2, rc 0, transno 4295069822 00000100:00100000:0.0:1785220581.317723:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510582912:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.317731:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b030a80/2, rc 0, transno 4295070419 00000100:00100000:0.0:1785220581.317734:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510582272:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.317793:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510586752:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.317796:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97840e00/2, rc 0, transno 4295070423 00000100:00100000:0.0:1785220581.317803:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97840e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510584704:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.317827:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.317911:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8441:1871938510586496:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.317929:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.317943:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.317992:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.318022:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510586624:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.318038:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.318067:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.318119:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901696 00000100:00100000:2.0:1785220581.318119:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901824 00000100:00100000:3.0:1785220581.318124:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901696 00000100:00100000:2.0:1785220581.318124:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901824 00010000:00010000:2.0:1785220581.318150:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ad6dbc9f/0xf62499ab927801d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d27 expref: 482 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.318182:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99958 00000100:00100000:2.0:1785220581.318192:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e13050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+482:2406:x1871938519901824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.318203:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.318205:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000043a32727/0xf62499ab927800ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c94 expref: 482 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318208:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ad6dbc9f/0xf62499ab927801d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d27 expref: 482 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.318228:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99959 00010000:00010000:2.0:1785220581.318230:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ad6dbc9f/0xf62499ab927801d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d27 expref: 482 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.318235:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+482:2406:x1871938519901696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.318243:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.318246:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000043a32727/0xf62499ab927800ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c94 expref: 482 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318254:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad6dbc9f/0xf62499ab927801d2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d27 expref: 482 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.318255:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000043a32727/0xf62499ab927800ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c94 expref: 482 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.318267:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000043a32727/0xf62499ab927800ba lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c94 expref: 481 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318268:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7183:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.318273:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7183:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.318277:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7182:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.318278:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.318280:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7182:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.318284:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318287:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.318292:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.318306:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.318306:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.318509:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e13050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+480:2406:x1871938519901824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 318us (440us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.318516:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99958 00010000:00010000:2.0:1785220581.318538:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318546:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318555:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318562:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318577:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.318645:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+479:2406:x1871938519901696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 409us (652us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.318651:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99959 00000100:00100000:2.0:1785220581.318683:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+972:3655:x1871938510586240:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8954us (9087us total) trans 4295069827 rc 0/0 00000100:00100000:2.0:1785220581.318690:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284025 00000100:00100000:2.0:1785220581.318703:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510586752 00000100:00100000:2.0:1785220581.318706:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510586752 00000100:00100000:2.0:1785220581.318742:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284027 00000100:00100000:2.0:1785220581.318748:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1048:3653:x1871938510586752:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.318778:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29118 count 1 00010000:00010000:2.0:1785220581.318800:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318813:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008949340e/0xf62499ab92783a96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7982b expref: 428 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318824:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71be:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.318827:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.318831:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318902:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008949340e/0xf62499ab92783a96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7982b expref: 428 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318914:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008949340e/0xf62499ab92783a96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7982b expref: 428 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.318932:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510587136:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.318966:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.318978:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318986:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.318994:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319000:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319018:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.319107:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+972:3653:x1871938510586368:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8165us (8486us total) trans 4295069828 rc 0/0 00000100:00100000:2.0:1785220581.319113:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284026 00010000:00010000:3.0:1785220581.319126:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.319129:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.319133:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319133:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319141:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.319145:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319154:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319167:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319174:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a453c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.319192:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.319194:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.319196:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.319206:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.319209:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.319216:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.319227:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.319234:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a4535 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.319249:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.319251:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.319253:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.319278:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365ca80/2, rc 0, transno 4295069828 00000100:00100000:3.0:1785220581.319284:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510586368:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.320316:0:14172:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cdc6ba65/0xf62499ab9277fd02 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ae9 expref: 478 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.320334:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510585856:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.320355:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.320370:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.320377:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.320387:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.320395:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.320413:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.320537:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+972:3655:x1871938510584960:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18560us (19034us total) trans 4295069829 rc 0/0 00000100:00100000:3.0:1785220581.320544:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284023 00000100:00100000:3.0:1785220581.320607:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510587008:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.320638:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220581.320668:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b030380/2, rc 0, transno 4295069827 00000100:00100000:3.0:1785220581.320673:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510586240:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.320682:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba4e00/2, rc 0, transno 4295069829 00000100:00100000:3.0:1785220581.320685:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510584960:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.320694:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba5880/2, rc 0, transno 4295069826 00000100:00100000:3.0:1785220581.320697:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510585472:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.320708:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba5c00/2, rc 0, transno 4295070425 00000100:00100000:3.0:1785220581.320716:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510585088:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.320985:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1048:3652:x1871938510583552:12345-0@lo:6:osp-syn-0-0.0 Request processed in 25920us (26011us total) trans 4295070424 rc 0/0 00000100:00100000:1.0:1785220581.320994:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284015 00000100:00100000:1.0:1785220581.321013:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510587008 00000100:00100000:1.0:1785220581.321019:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510587008 00000100:00100000:1.0:1785220581.321055:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284028 00000100:00100000:1.0:1785220581.321062:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1048:3655:x1871938510587008:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.321083:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29120 count 1 00000100:00100000:0.0:1785220581.321458:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510587264:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.321501:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.321579:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321597:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d9578d24/0xf62499ab92783c33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798e1 expref: 428 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321606:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c0:0x0].0x0 since=32s v=(0 0) 00010000:00010000:1.0:1785220581.321611:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.321615:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321644:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d9578d24/0xf62499ab92783c33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798e1 expref: 428 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321657:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d9578d24/0xf62499ab92783c33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798e1 expref: 428 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.321668:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510587392:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.321729:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01300 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.321744:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.321750:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321757:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.321760:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321780:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321797:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321805:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4527 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x717f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.321812:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.321824:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.321825:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.321827:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.321837:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510587264 00000100:00100000:1.0:1785220581.321842:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510587264 00000100:00100000:1.0:1785220581.321873:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284029 00000100:00100000:1.0:1785220581.321879:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1049:3652:x1871938510587264:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.321893:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29121 count 1 00010000:00010000:1.0:1785220581.321905:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321917:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c0bc485e/0xf62499ab92783d52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7995f expref: 428 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321924:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c1:0x0].0x0 since=32s v=(0 0) 00010000:00010000:1.0:1785220581.321927:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.321930:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321948:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c0bc485e/0xf62499ab92783d52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7995f expref: 428 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.321957:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c0bc485e/0xf62499ab92783d52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7995f expref: 428 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.321968:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510587520:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.322007:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.322420:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97840000/2, rc 0, transno 4295070420 00000100:00100000:2.0:1785220581.322427:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97840000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510583680:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.322438:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97842300/2, rc 0, transno 4295070418 00000100:00100000:2.0:1785220581.322441:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510583936:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.322452:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6edc00/2, rc 0, transno 4295069824 00000100:00100000:2.0:1785220581.322455:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510582784:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.322468:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.322474:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510586880:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.322504:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.322649:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a9b480/2, rc 0, transno 4295070424 00000100:00100000:0.0:1785220581.322655:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a9b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510583552:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.322663:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220581.322669:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510587648:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.323972:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.323988:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588032:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.324115:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510587776:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.324521:0:14193:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510586880 00000100:00100000:1.0:1785220581.324521:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.324528:0:14193:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510586880 00000100:00100000:1.0:1785220581.324552:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588160:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.324565:0:14193:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510588032 00000100:00100000:2.0:1785220581.324569:0:14193:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510588032 00000100:00100000:1.0:1785220581.324573:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.324589:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.324595:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588544:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.324619:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.324630:0:14193:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510588160 00000100:00100000:2.0:1785220581.324633:0:14193:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510588160 00000100:00100000:1.0:1785220581.324647:0:14204:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510588544 00000100:00100000:1.0:1785220581.324650:0:14204:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510588544 00000100:00100000:2.0:1785220581.324655:0:14193:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284030 00000100:00100000:2.0:1785220581.324663:0:14193:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1050:3655:x1871938510586880:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.324670:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284031 00000100:00100000:1.0:1785220581.324675:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+974:3653:x1871938510588032:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.324687:0:14193:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29119 count 1 00002000:00080000:1.0:1785220581.324687:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29063 count 1 00010000:00010000:1.0:1785220581.324710:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.324711:0:14193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.324732:0:14193:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d46901f9/0xf62499ab92783bca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798b7 expref: 428 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.324732:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000176ea025/0xf62499ab92780559 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ed2 expref: 477 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.324741:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7187:0x0].0x0 since=35s v=(0 0) 00010000:00010000:2.0:1785220581.324742:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71bf:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.324746:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.324746:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7187:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.324751:0:14193:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.324752:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.324792:0:14193:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d46901f9/0xf62499ab92783bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798b7 expref: 428 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.324792:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000176ea025/0xf62499ab92780559 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ed2 expref: 477 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.324806:0:14193:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d46901f9/0xf62499ab92783bca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798b7 expref: 428 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.324806:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000176ea025/0xf62499ab92780559 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ed2 expref: 477 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.324819:0:14193:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510588672:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.324820:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510588800:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.324872:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.324887:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.324901:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510587776 00000100:00100000:0.0:1785220581.324906:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510587776 00000100:00100000:0.0:1785220581.324931:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284032 00000100:00100000:0.0:1785220581.324938:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+975:3653:x1871938510588160:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.324951:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29064 count 1 00010000:00010000:0.0:1785220581.324964:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.324977:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe0e2271/0xf62499ab92780663 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f42 expref: 477 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.324985:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7188:0x0].0x0 since=35s v=(0 0) 00010000:00010000:0.0:1785220581.324988:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7188:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.324991:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.325008:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fe0e2271/0xf62499ab92780663 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f42 expref: 477 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.325017:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fe0e2271/0xf62499ab92780663 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f42 expref: 477 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.325026:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510589312:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.325074:0:14193:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.325120:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284033 00000100:00100000:2.0:1785220581.325128:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+975:3653:x1871938510588544:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.325140:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29067 count 1 00010000:00010000:2.0:1785220581.325153:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.325164:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ba673103/0xf62499ab92780aed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78125 expref: 477 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.325171:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x718b:0x0].0x0 since=34s v=(0 0) 00010000:00010000:2.0:1785220581.325174:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.325177:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.325197:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ba673103/0xf62499ab92780aed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78125 expref: 477 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.325207:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ba673103/0xf62499ab92780aed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78125 expref: 477 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.325262:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284034 00000100:00100000:3.0:1785220581.325274:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+975:3654:x1871938510587776:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.325287:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29061 count 1 00010000:00010000:3.0:1785220581.325299:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.325310:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b523a92b/0xf62499ab92780361 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e00 expref: 477 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.325316:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7185:0x0].0x0 since=35s v=(0 0) 00010000:00010000:3.0:1785220581.325319:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7185:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.325322:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.325339:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b523a92b/0xf62499ab92780361 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e00 expref: 477 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.325348:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b523a92b/0xf62499ab92780361 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e00 expref: 477 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.325357:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510589568:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.325385:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510589440:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.325386:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.325404:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510587904:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.325407:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.325429:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.325459:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510588416:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.325464:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510587904 00000100:00100000:2.0:1785220581.325468:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510587904 00000100:00100000:3.0:1785220581.325479:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.325520:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588928:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.325536:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9ae74c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.325563:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510588288:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.325958:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.327297:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510588416 00000100:00100000:2.0:1785220581.327308:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510588416 00000100:00100000:1.0:1785220581.327432:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.327476:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.327529:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b033b80/2, rc 0, transno 4295070422 00000100:00100000:1.0:1785220581.327545:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510582144:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.327605:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.327612:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510589696:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.327659:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.327687:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510589056:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.327709:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.327949:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510588288 00000100:00100000:2.0:1785220581.327955:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510588288 00000100:00100000:2.0:1785220581.327986:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510588928 00000100:00100000:2.0:1785220581.327990:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510588928 00000100:00100000:2.0:1785220581.328008:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510587648 00000100:00100000:2.0:1785220581.328010:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510587648 00000100:00100000:2.0:1785220581.328025:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510589696 00000100:00100000:2.0:1785220581.328028:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510589696 00000100:00100000:2.0:1785220581.328043:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510589056 00000100:00100000:2.0:1785220581.328046:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510589056 00000100:00100000:2.0:1785220581.328064:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284035 00000100:00100000:2.0:1785220581.328072:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+979:3655:x1871938510587904:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.328121:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29062 count 1 00010000:00010000:2.0:1785220581.328146:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328166:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f6d44a4d/0xf62499ab9278043a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e69 expref: 477 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328174:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7186:0x0].0x0 since=35s v=(0 0) 00010000:00010000:2.0:1785220581.328179:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7186:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.328184:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328226:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f6d44a4d/0xf62499ab9278043a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e69 expref: 477 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.328232:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284036 00010000:00010000:2.0:1785220581.328240:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f6d44a4d/0xf62499ab9278043a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e69 expref: 477 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.328242:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+979:3655:x1871938510588416:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.328258:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0001_UUID:14195:1871938510589824:192.168.201.51@tcp:104: 00002000:00080000:1.0:1785220581.328258:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29066 count 1 00010000:00010000:1.0:1785220581.328270:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.328282:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004fa3ca8a/0xf62499ab927809ea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78092 expref: 477 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.328288:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x718a:0x0].0x0 since=35s v=(0 0) 00010000:00010000:1.0:1785220581.328291:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.328294:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.328316:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004fa3ca8a/0xf62499ab927809ea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78092 expref: 477 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.328317:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.328325:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004fa3ca8a/0xf62499ab927809ea lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78092 expref: 477 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.328334:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510589952:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.328346:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284037 00000100:00100000:2.0:1785220581.328352:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+979:3654:x1871938510588288:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.328356:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01540 going to sleep for 11 seconds 00002000:00080000:2.0:1785220581.328365:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29065 count 1 00010000:00010000:2.0:1785220581.328376:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328387:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa420f1f/0xf62499ab927807dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77fd5 expref: 477 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328393:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7189:0x0].0x0 since=35s v=(0 0) 00010000:00010000:2.0:1785220581.328396:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7189:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.328399:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328415:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fa420f1f/0xf62499ab927807dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77fd5 expref: 477 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328423:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fa420f1f/0xf62499ab927807dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77fd5 expref: 477 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.328432:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510590080:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.328455:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.328472:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284038 00000100:00100000:2.0:1785220581.328479:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1053:3653:x1871938510588928:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.328492:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29122 count 1 00010000:00010000:2.0:1785220581.328505:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328517:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e1c545da/0xf62499ab92783e24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799cf expref: 428 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328524:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c2:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.328526:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.328530:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328549:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e1c545da/0xf62499ab92783e24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799cf expref: 428 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328559:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e1c545da/0xf62499ab92783e24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799cf expref: 428 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.328569:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510590208:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.328592:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.328608:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284039 00000100:00100000:2.0:1785220581.328616:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+979:3652:x1871938510587648:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.328630:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29060 count 1 00010000:00010000:2.0:1785220581.328641:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328653:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006cc1be7b/0xf62499ab927802ab lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d9e expref: 477 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328659:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7184:0x0].0x0 since=35s v=(0 0) 00010000:00010000:2.0:1785220581.328662:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7184:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.328665:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328683:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006cc1be7b/0xf62499ab927802ab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d9e expref: 477 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328693:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006cc1be7b/0xf62499ab927802ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d9e expref: 477 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.328702:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510590336:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.328736:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.328752:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284040 00000100:00100000:2.0:1785220581.328760:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1053:3653:x1871938510589696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.328783:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29125 count 1 00010000:00010000:2.0:1785220581.328796:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328812:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e8dac708/0xf62499ab92783ff2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ab6 expref: 428 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328818:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c5:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.328821:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.328825:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328842:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e8dac708/0xf62499ab92783ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ab6 expref: 428 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328851:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e8dac708/0xf62499ab92783ff2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ab6 expref: 428 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.328860:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510590464:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.328884:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920c00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.328903:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284041 00000100:00100000:2.0:1785220581.328910:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1053:3654:x1871938510589056:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.328922:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29123 count 1 00010000:00010000:2.0:1785220581.328932:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328943:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b669de8a/0xf62499ab92783eb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a0e expref: 428 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328956:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c3:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.328959:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.328963:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328982:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b669de8a/0xf62499ab92783eb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a0e expref: 428 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.328991:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b669de8a/0xf62499ab92783eb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a0e expref: 428 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.329001:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510590592:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.329023:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777b40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.329048:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510589184:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.329080:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.329163:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510589184 00000100:00100000:2.0:1785220581.329168:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510589184 00000100:00100000:2.0:1785220581.329199:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284042 00000100:00100000:2.0:1785220581.329205:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1054:3655:x1871938510589184:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.329277:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29124 count 1 00010000:00010000:2.0:1785220581.329289:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.329301:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000089efc31f/0xf62499ab92783f90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a7e expref: 428 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.329313:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c4:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.329316:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.329320:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.329339:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000089efc31f/0xf62499ab92783f90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a7e expref: 428 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.329348:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000089efc31f/0xf62499ab92783f90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a7e expref: 428 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.329358:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510590720:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.329389:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777e40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.330779:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.330823:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.330844:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519901952 00000100:00100000:0.0:1785220581.330850:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519901952 00010000:00010000:0.0:1785220581.330881:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008949340e/0xf62499ab92783a96 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7982b expref: 429 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.330909:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902080 00000100:00100000:0.0:1785220581.330913:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902080 00010000:00010000:0.0:1785220581.330928:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d9578d24/0xf62499ab92783c33 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798e1 expref: 430 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.330931:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99960 00000100:00100000:3.0:1785220581.330943:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+430:2406:x1871938519901952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.330946:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99961 00000100:00100000:0.0:1785220581.330951:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+430:2407:x1871938519902080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.330957:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.330957:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.330963:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008949340e/0xf62499ab92783a96 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7982b expref: 430 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.330963:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d9578d24/0xf62499ab92783c33 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798e1 expref: 430 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.330976:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008949340e/0xf62499ab92783a96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7982b expref: 430 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.330976:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d9578d24/0xf62499ab92783c33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798e1 expref: 430 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331000:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.331000:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.331003:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.331004:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.331010:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.331010:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331021:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.331021:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331041:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.331041:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.331080:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+430:2406:x1871938519901952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 139us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.331105:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99960 00010000:00010000:2.0:1785220581.331141:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008949340e/0xf62499ab92783a96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7982b expref: 429 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.331157:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510587136:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.331178:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.331188:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.331201:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.331212:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.331252:0:13836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d9578d24/0xf62499ab92783c33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798e1 expref: 428 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.331263:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510587392:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.331277:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.331285:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.331292:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.331300:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.331311:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.331335:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.331407:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.331423:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.331511:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.331517:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331524:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.331528:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331536:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331549:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331556:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a454a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331573:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.331576:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331582:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffa00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.331585:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331596:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331605:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331611:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a4543 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.331623:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.331624:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.331626:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.331667:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2407:x1871938519902080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 714us (842us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.331675:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99961 00000100:00100000:0.0:1785220581.331787:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.331806:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902208 00000100:00100000:0.0:1785220581.331812:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902208 00010000:00010000:0.0:1785220581.331831:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c0bc485e/0xf62499ab92783d52 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7995f expref: 427 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.331852:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99962 00000100:00100000:0.0:1785220581.331860:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2406:x1871938519902208:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.331869:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.331874:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c0bc485e/0xf62499ab92783d52 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7995f expref: 427 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.331884:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c0bc485e/0xf62499ab92783d52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7995f expref: 427 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.331898:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.331902:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.331906:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.331914:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.331927:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.331947:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2406:x1871938519902208:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (160us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.331954:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99962 00000100:00100000:2.0:1785220581.332788:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1056:3653:x1871938510586752:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14037us (14958us total) trans 4295070427 rc 0/0 00000100:00100000:2.0:1785220581.332798:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284027 00010000:00010000:2.0:1785220581.332825:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.333262:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.333318:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510589312:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.333340:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.333371:0:14193:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510588672:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.333388:0:14193:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.333427:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba7100/2, rc 0, transno 4295070427 00000100:00100000:0.0:1785220581.333449:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510586752:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.333471:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902464 00000100:00100000:0.0:1785220581.333476:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902464 00010000:00010000:0.0:1785220581.333502:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fe0e2271/0xf62499ab92780663 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f42 expref: 478 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.333540:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99963 00000100:00100000:0.0:1785220581.333549:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+478:2407:x1871938519902464:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.333562:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.333567:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fe0e2271/0xf62499ab92780663 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f42 expref: 478 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.333579:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fe0e2271/0xf62499ab92780663 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f42 expref: 478 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.333600:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe0e2271/0xf62499ab92780663 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77f42 expref: 478 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.333614:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7188:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.333618:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7188:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.333623:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.333632:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.333657:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.333695:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+477:2407:x1871938519902464:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 147us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.333702:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99963 00010000:00010000:2.0:1785220581.333949:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.333962:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.333972:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.333981:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.334000:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.334171:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+980:3653:x1871938510588160:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9233us (9595us total) trans 4295069830 rc 0/0 00000100:00100000:2.0:1785220581.334180:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284032 00000100:00100000:0.0:1785220581.334668:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510590848:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.334672:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1055:3655:x1871938510587008:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13609us (14032us total) trans 4295070426 rc 0/0 00000100:00100000:1.0:1785220581.334680:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284028 00000100:00100000:0.0:1785220581.334705:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.334720:0:14232:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0bc485e/0xf62499ab92783d52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7995f expref: 426 pid: 14203 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.334734:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510587520:192.168.201.51@tcp:104: 00000004:00080000:0.0:1785220581.334734:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365e680/2, rc 0, transno 4295069830 00000100:00100000:0.0:1785220581.334745:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588160:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.334753:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.334762:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.334783:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.334792:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.334800:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.334812:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510590848 00000100:00100000:2.0:1785220581.334817:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510590848 00010000:00010000:1.0:1785220581.334822:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.334860:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284043 00000100:00100000:2.0:1785220581.334868:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d935a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1055:3653:x1871938510590848:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.334890:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29126 count 1 00010000:00010000:2.0:1785220581.334910:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.334916:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1056:3652:x1871938510587264:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13036us (13414us total) trans 4295070428 rc 0/0 00000100:00100000:1.0:1785220581.334921:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284029 00010000:00010000:2.0:1785220581.334926:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000edcca853/0xf62499ab927840c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b26 expref: 425 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.334934:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c6:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.334938:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.334943:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.334948:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510588800:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.334960:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.334967:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000edcca853/0xf62499ab927840c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b26 expref: 425 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.334979:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000edcca853/0xf62499ab927840c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b26 expref: 425 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.334990:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510590976:192.168.201.51@tcp:104: 00000004:00080000:1.0:1785220581.335016:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba4380/2, rc 0, transno 4295070428 00000100:00100000:1.0:1785220581.335026:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510587264:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.335037:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.335056:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510591232:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.335056:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510591104:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.335079:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.335119:0:14185:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510591232 00000100:00100000:1.0:1785220581.335123:0:14185:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510591232 00000004:00080000:2.0:1785220581.335136:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba5f80/2, rc 0, transno 4295070426 00000100:00100000:2.0:1785220581.335145:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510587008:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.335159:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284044 00000100:00100000:1.0:1785220581.335166:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1056:3655:x1871938510591232:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.335178:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29127 count 1 00010000:00010000:1.0:1785220581.335190:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.335202:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000094f1028a/0xf62499ab927841b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bc7 expref: 425 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.335208:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c7:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.335212:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.335216:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.335234:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000094f1028a/0xf62499ab927841b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bc7 expref: 425 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.335243:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000094f1028a/0xf62499ab927841b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bc7 expref: 425 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.335252:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0000_UUID:14185:1871938510591360:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.335280:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.335341:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.335351:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510591488:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.335374:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.335396:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.335534:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510591488 00000100:00100000:2.0:1785220581.335538:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510591488 00000100:00100000:2.0:1785220581.335560:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284045 00000100:00100000:2.0:1785220581.335566:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1057:3653:x1871938510591488:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.335577:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29128 count 1 00010000:00010000:2.0:1785220581.335588:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.335600:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000df1fae8e/0xf62499ab92784317 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c3e expref: 425 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.335602:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.335610:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.335611:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c8:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.335615:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.335619:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ada00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.335619:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.335623:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.335632:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.335637:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000df1fae8e/0xf62499ab92784317 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c3e expref: 425 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.335645:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000df1fae8e/0xf62499ab92784317 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c3e expref: 425 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.335650:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.335655:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510591616:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.335659:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4566 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7188:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.335677:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9ae74c00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.335680:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.335684:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.335690:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:2.0:1785220581.335691:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510591104 00010000:00010000:3.0:1785220581.335694:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.335694:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510591104 00010000:00010000:3.0:1785220581.335701:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.335711:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.335717:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284046 00010000:00010000:3.0:1785220581.335719:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a4551 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.335723:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+980:3654:x1871938510591104:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.335732:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00080000:2.0:1785220581.335734:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29068 count 1 00010000:00010000:3.0:1785220581.335735:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.335736:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.335746:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.335757:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d5ef18c8/0xf62499ab92780c52 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b781b1 expref: 476 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.335788:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x718c:0x0].0x0 since=34s v=(0 0) 00010000:00010000:2.0:1785220581.335792:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.335795:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.335815:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d5ef18c8/0xf62499ab92780c52 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b781b1 expref: 476 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.335819:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510589568:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.335825:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d5ef18c8/0xf62499ab92780c52 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b781b1 expref: 476 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.335838:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.335846:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510591744:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.335869:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.335922:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.335944:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.337974:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510589440:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.338006:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.339574:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902592 00000100:00100000:2.0:1785220581.339587:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902592 00010000:00010000:2.0:1785220581.339627:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d46901f9/0xf62499ab92783bca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798b7 expref: 426 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.339674:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99964 00000100:00100000:2.0:1785220581.339683:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+426:2407:x1871938519902592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.339698:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.339703:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d46901f9/0xf62499ab92783bca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798b7 expref: 426 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.339714:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d46901f9/0xf62499ab92783bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798b7 expref: 426 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.339736:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d46901f9/0xf62499ab92783bca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798b7 expref: 426 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.339752:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.339756:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.339762:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.339782:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.339811:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.339838:0:14193:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.339849:0:14193:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.339859:0:14193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.339867:0:14193:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.339884:0:14193:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.340014:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+425:2407:x1871938519902592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 330us (4091us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.340022:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99964 00010000:00010000:2.0:1785220581.340483:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.340490:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.340508:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.340515:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.340525:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.340542:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.340553:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a4558 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.340577:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.340578:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.340580:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.340951:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.341023:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.341164:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.341220:0:14193:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1058:3655:x1871938510586880:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16554us (18712us total) trans 4295070429 rc 0/0 00000100:00100000:3.0:1785220581.341230:0:14193:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284030 00000100:00100000:1.0:1785220581.341325:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510589952:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.341347:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.341356:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902336 00000100:00100000:2.0:1785220581.341362:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902336 00010000:00010000:2.0:1785220581.341385:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000176ea025/0xf62499ab92780559 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ed2 expref: 477 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.341411:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902720 00000100:00100000:2.0:1785220581.341414:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902720 00010000:00010000:2.0:1785220581.341427:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b523a92b/0xf62499ab92780361 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e00 expref: 478 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.341433:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99965 00000100:00100000:3.0:1785220581.341444:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+478:2406:x1871938519902336:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.341457:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.341460:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99966 00010000:00010000:3.0:1785220581.341462:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000176ea025/0xf62499ab92780559 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ed2 expref: 478 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.341466:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90701c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+478:2405:x1871938519902720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.341474:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000176ea025/0xf62499ab92780559 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ed2 expref: 478 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.341491:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.341492:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000176ea025/0xf62499ab92780559 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77ed2 expref: 478 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.341497:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b523a92b/0xf62499ab92780361 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e00 expref: 478 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.341505:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7187:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.341506:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b523a92b/0xf62499ab92780361 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e00 expref: 477 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.341509:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7187:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.341515:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.341517:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b523a92b/0xf62499ab92780361 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e00 expref: 477 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.341525:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.341528:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7185:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.341531:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7185:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.341534:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.341541:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.341545:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.341556:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.341612:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.341614:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90701c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+476:2405:x1871938519902720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 147us (590us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.341621:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99966 00010000:00010000:1.0:1785220581.341621:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.341632:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902848 00000100:00100000:2.0:1785220581.341635:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902848 00010000:00010000:1.0:1785220581.341636:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.341644:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.341646:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ba673103/0xf62499ab92780aed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78125 expref: 476 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.341661:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.341726:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+476:2406:x1871938519902336:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (626us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.341735:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99965 00000100:00100000:1.0:1785220581.341835:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+981:3653:x1871938510588032:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17158us (17314us total) trans 4295069831 rc 0/0 00000100:00100000:1.0:1785220581.341842:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284031 00000100:00100000:0.0:1785220581.342449:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.342469:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519903104 00000100:00100000:0.0:1785220581.342473:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519903104 00010000:00010000:0.0:1785220581.342493:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004fa3ca8a/0xf62499ab927809ea lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78092 expref: 476 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.342775:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99967 00000100:00100000:2.0:1785220581.342786:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+476:2407:x1871938519902848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.342798:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.342804:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ba673103/0xf62499ab92780aed lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78125 expref: 476 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.342815:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ba673103/0xf62499ab92780aed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78125 expref: 476 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.342833:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ba673103/0xf62499ab92780aed lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78125 expref: 476 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.342900:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365fb80/2, rc 0, transno 4295069831 00010000:00010000:2.0:1785220581.342902:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.342906:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.342911:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.342912:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588032:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.342919:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.342939:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.342970:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+475:2407:x1871938519902848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 187us (1807us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.342977:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99967 00000100:00100000:1.0:1785220581.342979:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510591872:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.342995:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.343001:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343009:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:1.0:1785220581.343010:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.343012:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343021:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343032:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343039:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a455f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7187:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343056:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.343058:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.343058:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.343060:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.343079:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510590080:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.343111:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.343313:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510590208:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.343328:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343338:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.343352:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510590592:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.343366:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343382:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.343387:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510590336:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.343398:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343405:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.343420:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510590464:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.343424:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.343431:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343441:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.343454:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510590720:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.343465:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343477:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0000_UUID:14185:1871938510591360:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.343482:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343489:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343496:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343506:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343514:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343519:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.343529:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.343560:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.343575:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.343588:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519903744 00000100:00100000:1.0:1785220581.343593:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519903744 00010000:00010000:1.0:1785220581.343610:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000089efc31f/0xf62499ab92783f90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a7e expref: 425 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343628:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519903360 00000100:00100000:1.0:1785220581.343630:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519903360 00010000:00010000:1.0:1785220581.343641:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e8dac708/0xf62499ab92783ff2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ab6 expref: 426 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343656:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519903488 00000100:00100000:1.0:1785220581.343659:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519903488 00000100:00100000:2.0:1785220581.343661:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+981:3653:x1871938510588544:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18534us (19041us total) trans 4295069832 rc 0/0 00000100:00100000:2.0:1785220581.343669:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284033 00010000:00010000:1.0:1785220581.343669:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b669de8a/0xf62499ab92783eb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a0e expref: 427 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343679:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519903872 00000100:00100000:1.0:1785220581.343681:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519903872 00000100:00100000:2.0:1785220581.343682:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510591872 00000100:00100000:2.0:1785220581.343685:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510591872 00010000:00010000:1.0:1785220581.343690:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fa420f1f/0xf62499ab927807dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77fd5 expref: 475 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343700:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519902976 00000100:00100000:1.0:1785220581.343703:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519902976 00000100:00100000:2.0:1785220581.343705:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284047 00000100:00100000:2.0:1785220581.343710:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+981:3653:x1871938510591872:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.343713:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f6d44a4d/0xf62499ab9278043a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e69 expref: 476 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343727:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519903232 00002000:00080000:2.0:1785220581.343729:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29069 count 1 00000100:00100000:1.0:1785220581.343729:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519903232 00010000:00010000:1.0:1785220581.343737:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006cc1be7b/0xf62499ab927802ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d9e expref: 477 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343746:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343746:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519903616 00000100:00100000:1.0:1785220581.343748:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519903616 00010000:00010000:1.0:1785220581.343757:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e1c545da/0xf62499ab92783e24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799cf expref: 428 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343760:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000020dfd67b/0xf62499ab92780d2b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7821a expref: 477 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343783:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x718d:0x0].0x0 since=34s v=(0 0) 00010000:00010000:2.0:1785220581.343787:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718d:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.343787:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904256 00000100:00100000:1.0:1785220581.343789:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904256 00010000:00010000:2.0:1785220581.343791:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343800:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d5ef18c8/0xf62499ab92780c52 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b781b1 expref: 478 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343809:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904128 00000100:00100000:1.0:1785220581.343811:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904128 00010000:00010000:2.0:1785220581.343815:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000020dfd67b/0xf62499ab92780d2b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7821a expref: 478 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343820:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000df1fae8e/0xf62499ab92784317 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c3e expref: 429 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343828:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000020dfd67b/0xf62499ab92780d2b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7821a expref: 478 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.343831:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99968 00000100:00100000:1.0:1785220581.343837:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+429:2405:x1871938519903744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.343845:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.343850:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000089efc31f/0xf62499ab92783f90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a7e expref: 429 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343865:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000089efc31f/0xf62499ab92783f90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a7e expref: 429 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343881:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089efc31f/0xf62499ab92783f90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a7e expref: 429 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343892:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.343896:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.343900:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.343902:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510592000:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.343908:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343924:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.343939:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+428:2405:x1871938519903744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (881us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.343944:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777e40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.343945:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99968 00000100:00100000:1.0:1785220581.343958:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99969 00000100:00100000:1.0:1785220581.343964:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9365c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2405:x1871938519903360:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.343969:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510591616:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.343971:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.343976:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e8dac708/0xf62499ab92783ff2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ab6 expref: 427 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.343980:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.343992:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e8dac708/0xf62499ab92783ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ab6 expref: 427 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.344002:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510591744:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.344005:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8dac708/0xf62499ab92783ff2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ab6 expref: 427 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344012:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344014:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344017:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344021:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344025:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344028:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344033:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344044:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.344046:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344053:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.344058:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9365c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+426:2405:x1871938519903360:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (720us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.344064:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99969 00010000:00010000:2.0:1785220581.344067:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.344076:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99970 00000100:00100000:1.0:1785220581.344082:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+425:2406:x1871938519903488:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.344177:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1058:3655:x1871938510589184:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14971us (15095us total) trans 4295070430 rc 0/0 00000100:00100000:2.0:1785220581.344183:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284042 00010000:00010000:2.0:1785220581.344202:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.344207:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344274:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.344280:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b669de8a/0xf62499ab92783eb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a0e expref: 425 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344294:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b669de8a/0xf62499ab92783eb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a0e expref: 425 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344308:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b669de8a/0xf62499ab92783eb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a0e expref: 425 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344317:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344320:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344324:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344331:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344347:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.344366:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+424:2406:x1871938519903488:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 284us (983us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.344373:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99970 00000100:00100000:1.0:1785220581.344386:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99971 00000100:00100000:1.0:1785220581.344392:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+478:2406:x1871938519903872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.344400:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.344404:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fa420f1f/0xf62499ab927807dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77fd5 expref: 478 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344419:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fa420f1f/0xf62499ab927807dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77fd5 expref: 478 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344432:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa420f1f/0xf62499ab927807dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77fd5 expref: 478 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344445:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7189:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344448:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7189:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344452:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344458:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344473:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.344492:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+477:2406:x1871938519903872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (1086us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.344498:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99971 00000100:00100000:1.0:1785220581.344511:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99972 00000100:00100000:1.0:1785220581.344517:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+476:2406:x1871938519902976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.344524:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.344529:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f6d44a4d/0xf62499ab9278043a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e69 expref: 476 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344543:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f6d44a4d/0xf62499ab9278043a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e69 expref: 476 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344556:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.344556:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7186:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344559:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7186:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.344561:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344563:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344570:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344571:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344583:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344584:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.344590:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a456d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.344600:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+476:2406:x1871938519902976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (1174us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.344606:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99972 00010000:00010000:2.0:1785220581.344607:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.344610:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344616:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffe00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:1.0:1785220581.344617:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99973 00010000:00010000:2.0:1785220581.344619:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.344623:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+475:2406:x1871938519903232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.344627:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344630:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.344634:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006cc1be7b/0xf62499ab927802ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d9e expref: 475 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344638:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344644:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006cc1be7b/0xf62499ab927802ab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d9e expref: 475 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344645:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a45ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344658:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.344660:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.344661:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006cc1be7b/0xf62499ab927802ab lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77d9e expref: 475 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344662:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.344669:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7184:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344672:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7184:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.344673:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344677:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344682:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344685:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344695:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344701:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.344703:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344714:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.344717:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+474:2406:x1871938519903232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (1275us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.344723:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99973 00000100:00100000:1.0:1785220581.344734:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99974 00000100:00100000:1.0:1785220581.344740:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+423:2406:x1871938519903616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.344747:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.344751:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e1c545da/0xf62499ab92783e24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799cf expref: 423 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344760:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e1c545da/0xf62499ab92783e24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799cf expref: 423 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344791:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1c545da/0xf62499ab92783e24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799cf expref: 423 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344801:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344804:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344807:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344815:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.344820:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+982:3654:x1871938510588288:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16468us (18862us total) trans 4295069833 rc 0/0 00000100:00100000:2.0:1785220581.344826:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284037 00010000:00010000:1.0:1785220581.344831:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.344849:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+422:2406:x1871938519903616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (1329us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.344855:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99974 00000100:00100000:1.0:1785220581.344868:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99975 00000100:00100000:1.0:1785220581.344873:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+473:2406:x1871938519904256:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.344880:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.344886:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d5ef18c8/0xf62499ab92780c52 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b781b1 expref: 473 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344896:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d5ef18c8/0xf62499ab92780c52 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b781b1 expref: 473 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344914:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d5ef18c8/0xf62499ab92780c52 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b781b1 expref: 473 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344918:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.344923:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344925:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.344928:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.344930:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.344932:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344934:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344941:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344943:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.344953:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.344957:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.344960:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f9e153b/0xf62499ab927a4589 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7189:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.344970:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+472:2406:x1871938519904256:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (1410us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.344976:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.344977:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.344977:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99975 00010000:00010000:2.0:1785220581.344979:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.344988:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.344989:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99976 00000100:00100000:1.0:1785220581.344995:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+421:2405:x1871938519904128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.344997:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345002:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.345005:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345007:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000df1fae8e/0xf62499ab92784317 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c3e expref: 421 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.345013:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345017:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000df1fae8e/0xf62499ab92784317 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c3e expref: 421 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.345025:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.345034:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df1fae8e/0xf62499ab92784317 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c3e expref: 421 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345043:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.345046:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.345050:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345057:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345071:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.345192:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+420:2405:x1871938519904128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (1509us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.345199:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99976 00010000:00010000:1.0:1785220581.345220:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345228:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345236:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345243:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345256:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.345396:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99977 00000100:00100000:0.0:1785220581.345408:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+471:2406:x1871938519903104:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.345416:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.345421:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004fa3ca8a/0xf62499ab927809ea lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78092 expref: 471 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.345435:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004fa3ca8a/0xf62499ab927809ea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78092 expref: 471 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.345448:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004fa3ca8a/0xf62499ab927809ea lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78092 expref: 471 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.345458:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.345461:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.345465:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.345473:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.345492:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.345683:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:0.0:1785220581.345711:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+470:2406:x1871938519903104:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 307us (3262us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.345717:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.345718:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99977 00010000:00010000:3.0:1785220581.345725:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.345728:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345735:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345751:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345758:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a45a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345784:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.345788:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345795:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.345798:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345804:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345813:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.345820:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a4590 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.345864:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1059:3653:x1871938510588928:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17385us (18433us total) trans 4295070432 rc 0/0 00000100:00100000:1.0:1785220581.345871:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284038 00010000:00010000:1.0:1785220581.345892:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345901:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345910:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.345917:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.345923:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1058:3654:x1871938510589056:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17013us (18213us total) trans 4295070431 rc 0/0 00000100:00100000:2.0:1785220581.345930:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284041 00010000:00010000:3.0:1785220581.345938:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.345964:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.345989:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.346080:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+982:3655:x1871938510588416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17841us (20599us total) trans 4295069834 rc 0/0 00000100:00100000:0.0:1785220581.346083:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.346114:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284036 00000100:00100000:0.0:1785220581.346128:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904000 00000100:00100000:0.0:1785220581.346146:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904000 00010000:00010000:1.0:1785220581.346154:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346161:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346167:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000094f1028a/0xf62499ab927841b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bc7 expref: 420 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346175:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346182:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346191:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.346191:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904384 00000100:00100000:0.0:1785220581.346194:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904384 00010000:00010000:0.0:1785220581.346203:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000edcca853/0xf62499ab927840c4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b26 expref: 421 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.346265:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1058:3653:x1871938510589696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17505us (18604us total) trans 4295070433 rc 0/0 00000100:00100000:1.0:1785220581.346271:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284040 00010000:00010000:1.0:1785220581.346287:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.346292:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346298:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.346301:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346309:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346320:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346328:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4582 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346344:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.346347:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346353:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fea00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:0.0:1785220581.346353:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99978 00010000:00010000:1.0:1785220581.346356:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.346361:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+421:2406:x1871938519904000:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.346364:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346370:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.346373:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346375:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000094f1028a/0xf62499ab927841b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bc7 expref: 421 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346382:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a459e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346386:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000094f1028a/0xf62499ab927841b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bc7 expref: 421 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346394:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.346396:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.346398:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.346399:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094f1028a/0xf62499ab927841b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bc7 expref: 421 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346408:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.346411:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.346416:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346423:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.346430:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa2300/2, rc 0, transno 4295070433 00000100:00100000:1.0:1785220581.346441:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510589696:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.346444:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1785220581.346452:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba5180/2, rc 0, transno 4295070432 00000100:00100000:1.0:1785220581.346456:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588928:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.346458:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+420:2406:x1871938519904000:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.346464:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99978 00000004:00080000:1.0:1785220581.346467:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365d880/2, rc 0, transno 4295069832 00000100:00100000:1.0:1785220581.346476:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510588544:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.346476:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99979 00000100:00100000:0.0:1785220581.346481:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+419:2407:x1871938519904384:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.346489:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.346492:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346493:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000edcca853/0xf62499ab927840c4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b26 expref: 419 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346501:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346503:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000edcca853/0xf62499ab927840c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b26 expref: 419 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346509:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346514:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.346517:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346517:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.346521:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346528:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.346529:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.346540:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.346553:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+419:2407:x1871938519904384:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 72us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.346561:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99979 00000100:00100000:1.0:1785220581.346603:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1058:3655:x1871938510591232:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11437us (11522us total) trans 4295070434 rc 0/0 00000100:00100000:1.0:1785220581.346609:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284044 00010000:00010000:1.0:1785220581.346625:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.346630:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346638:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.346641:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346650:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346661:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346668:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a45ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.346682:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.346684:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.346685:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.346778:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.346791:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510592384:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.346822:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.346853:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510592128:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.346876:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.346898:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba7b80/2, rc 0, transno 4295070431 00000100:00100000:1.0:1785220581.346906:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510589056:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.346934:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510592512:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.346955:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.346983:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510592384 00000100:00100000:1.0:1785220581.346988:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510592384 00000100:00100000:1.0:1785220581.347012:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510592128 00000100:00100000:1.0:1785220581.347014:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510592128 00000100:00100000:1.0:1785220581.347028:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510592512 00000100:00100000:1.0:1785220581.347030:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510592512 00000100:00100000:1.0:1785220581.347045:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284048 00000100:00100000:1.0:1785220581.347051:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+982:3653:x1871938510592384:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.347072:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29070 count 1 00010000:00010000:1.0:1785220581.347115:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347129:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000646eab87/0xf62499ab92780df6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78283 expref: 469 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347135:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x718e:0x0].0x0 since=34s v=(0 0) 00010000:00010000:1.0:1785220581.347139:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.347143:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347163:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000646eab87/0xf62499ab92780df6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78283 expref: 469 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347173:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000646eab87/0xf62499ab92780df6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78283 expref: 469 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.347185:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510592640:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.347222:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01900 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.347241:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284049 00000100:00100000:1.0:1785220581.347249:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1059:3654:x1871938510592128:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.347260:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29129 count 1 00010000:00010000:1.0:1785220581.347272:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347289:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006ef89ed9/0xf62499ab927843f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79cbc expref: 418 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347296:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71c9:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.347299:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.347302:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347321:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006ef89ed9/0xf62499ab927843f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79cbc expref: 418 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347330:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006ef89ed9/0xf62499ab927843f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79cbc expref: 418 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.347340:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510592768:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.347362:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01d80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.347393:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284050 00000100:00100000:1.0:1785220581.347399:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1059:3653:x1871938510592512:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.347412:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29131 count 1 00010000:00010000:1.0:1785220581.347423:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347435:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006e777c0a/0xf62499ab92784620 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79dbf expref: 418 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347447:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71cb:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.347449:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.347627:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347649:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006e777c0a/0xf62499ab92784620 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79dbf expref: 418 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.347658:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006e777c0a/0xf62499ab92784620 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79dbf expref: 418 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.347667:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0000_UUID:14183:1871938510592896:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.347705:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df019c0 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.347727:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365ea00/2, rc 0, transno 4295069833 00000100:00100000:1.0:1785220581.347736:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510588288:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.347745:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.347750:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510592256:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.347783:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.347803:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510592256 00000100:00100000:0.0:1785220581.347807:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510592256 00000100:00100000:1.0:1785220581.347832:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510593024:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.347832:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284051 00000100:00100000:0.0:1785220581.347838:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1060:3654:x1871938510592256:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.347856:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:0.0:1785220581.347921:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29130 count 1 00010000:00010000:0.0:1785220581.347935:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.347947:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000042bf09e6/0xf62499ab927844e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d33 expref: 418 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.347953:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ca:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.347956:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.347960:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.347979:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000042bf09e6/0xf62499ab927844e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d33 expref: 418 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.347987:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000042bf09e6/0xf62499ab927844e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d33 expref: 418 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.347995:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:13867:1871938510593152:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.348029:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d87b9b300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.348044:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510593024 00000100:00100000:0.0:1785220581.348048:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510593024 00000100:00100000:0.0:1785220581.348073:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284052 00000100:00100000:0.0:1785220581.348079:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+983:3654:x1871938510593024:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.348122:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29071 count 1 00010000:00010000:0.0:1785220581.348139:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.348150:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f126e085/0xf62499ab92780fbd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7832b expref: 469 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.348156:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x718f:0x0].0x0 since=34s v=(0 0) 00010000:00010000:0.0:1785220581.348159:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.348162:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.348180:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f126e085/0xf62499ab92780fbd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7832b expref: 469 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.348188:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f126e085/0xf62499ab92780fbd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7832b expref: 469 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.348198:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510593280:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.348384:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdd80 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.349365:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349385:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349395:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349402:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349428:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.349590:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+984:3652:x1871938510587648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20973us (22111us total) trans 4295069835 rc 0/0 00000100:00100000:0.0:1785220581.349599:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284039 00010000:00010000:0.0:1785220581.349631:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349639:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349647:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349653:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349666:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.349756:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1061:3653:x1871938510591488:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14189us (14381us total) trans 4295070435 rc 0/0 00000100:00100000:0.0:1785220581.349763:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284045 00010000:00010000:0.0:1785220581.349798:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349807:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349815:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349822:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.349830:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.349903:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+984:3654:x1871938510591104:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14179us (14506us total) trans 4295069836 rc 0/0 00000100:00100000:0.0:1785220581.349909:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284046 00010000:00010000:0.0:1785220581.349955:0:14195:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f6d44a4d/0xf62499ab9278043a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77e69 expref: 469 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.349970:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0001_UUID:14195:1871938510589824:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.349993:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.350002:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.350009:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.350018:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.350025:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.350034:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.350131:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+984:3655:x1871938510587904:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22059us (24700us total) trans 4295069837 rc 0/0 00000100:00100000:0.0:1785220581.350137:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284035 00000004:00080000:0.0:1785220581.350179:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365f100/2, rc 0, transno 4295069835 00000100:00100000:0.0:1785220581.350189:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510587648:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.350381:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.350388:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350402:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.350406:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350415:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350434:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350441:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4597 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350460:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.350464:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350470:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fee00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.350474:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350480:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350490:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350498:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a45c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350512:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.350515:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350521:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.350525:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350533:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350542:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350549:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a45c8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350563:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.350566:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350572:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fec00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.350576:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350583:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350593:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350600:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a457b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7186:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.350615:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.350617:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.350619:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.350645:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa0a80/2, rc 0, transno 4295070435 00000100:00100000:1.0:1785220581.350657:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510591488:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.350738:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.350749:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510593536:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.350818:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.350862:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa1500/2, rc 0, transno 4295069836 00000100:00100000:1.0:1785220581.350870:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510591104:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.350875:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.350882:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510593408:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.350908:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.350953:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b109500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510593664:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.350975:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.351036:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa3b80/2, rc 0, transno 4295070434 00000100:00100000:0.0:1785220581.351054:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510591232:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.351068:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365df80/2, rc 0, transno 4295069837 00000100:00100000:0.0:1785220581.351074:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510587904:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.351113:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365d180/2, rc 0, transno 4295069834 00000100:00100000:0.0:1785220581.351118:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510588416:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.351128:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba4700/2, rc 0, transno 4295070430 00000100:00100000:0.0:1785220581.351132:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510589184:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.351142:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba6d80/2, rc 0, transno 4295070429 00000100:00100000:0.0:1785220581.351145:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510586880:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.351350:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.351358:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510594048:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.351384:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.351432:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510594176:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.351453:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.351480:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510593920:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.351501:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.351517:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1785220581.351522:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510593792:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.351540:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.351552:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510594304:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.351569:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.351628:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510593536 00000100:00100000:0.0:1785220581.351634:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510593536 00000100:00100000:0.0:1785220581.351678:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510593408 00000100:00100000:0.0:1785220581.351682:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510593408 00000100:00100000:0.0:1785220581.351703:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510593664 00000100:00100000:0.0:1785220581.351706:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510593664 00000100:00100000:0.0:1785220581.351721:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510594048 00000100:00100000:0.0:1785220581.351724:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510594048 00000100:00100000:0.0:1785220581.351737:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510594176 00000100:00100000:0.0:1785220581.351740:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510594176 00000100:00100000:0.0:1785220581.351752:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510593920 00000100:00100000:0.0:1785220581.351755:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510593920 00000100:00100000:0.0:1785220581.351780:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510593792 00000100:00100000:0.0:1785220581.351783:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510593792 00000100:00100000:0.0:1785220581.351800:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510594304 00000100:00100000:0.0:1785220581.351803:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510594304 00000100:00100000:0.0:1785220581.351820:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284053 00000100:00100000:0.0:1785220581.351827:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+987:3653:x1871938510593536:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.351849:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29072 count 1 00010000:00010000:0.0:1785220581.351870:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.351887:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000097ca6708/0xf62499ab927810ea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783b7 expref: 468 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.351895:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7190:0x0].0x0 since=34s v=(0 0) 00010000:00010000:0.0:1785220581.351899:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7190:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.351903:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.351930:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000097ca6708/0xf62499ab927810ea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783b7 expref: 468 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.351943:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000097ca6708/0xf62499ab927810ea lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783b7 expref: 468 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.351955:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510594432:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.352010:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.352026:0:14186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284054 00000100:00100000:0.0:1785220581.352034:0:14186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1064:3654:x1871938510593408:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.352047:0:14186:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29132 count 1 00010000:00010000:0.0:1785220581.352061:0:14186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352074:0:14186:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b3c649a6/0xf62499ab9278479a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e36 expref: 418 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352081:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71cc:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.352107:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.352112:0:14186:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352133:0:14186:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b3c649a6/0xf62499ab9278479a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e36 expref: 418 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352143:0:14186:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b3c649a6/0xf62499ab9278479a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e36 expref: 418 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.352154:0:14186:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0000_UUID:14186:1871938510594560:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.352178:0:14186:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd180 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.352195:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284055 00000100:00100000:0.0:1785220581.352202:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+987:3653:x1871938510593664:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.352298:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29073 count 1 00010000:00010000:0.0:1785220581.352312:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352324:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000baf8d157/0xf62499ab927811bc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7842e expref: 468 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352330:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7191:0x0].0x0 since=34s v=(0 0) 00010000:00010000:0.0:1785220581.352333:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7191:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.352337:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352356:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000baf8d157/0xf62499ab927811bc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7842e expref: 468 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352366:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000baf8d157/0xf62499ab927811bc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7842e expref: 468 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.352377:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510594688:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.352400:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd840 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.352418:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284056 00000100:00100000:0.0:1785220581.352426:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1064:3653:x1871938510594048:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.352454:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29135 count 1 00010000:00010000:0.0:1785220581.352466:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352477:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003d200ee0/0xf62499ab92784a87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f9b expref: 418 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352483:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71cf:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.352485:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.352488:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352506:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003d200ee0/0xf62499ab92784a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f9b expref: 418 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.352514:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003d200ee0/0xf62499ab92784a87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f9b expref: 418 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.352528:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510594816:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.352551:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdc00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.352633:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284057 00000100:00100000:1.0:1785220581.352642:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+987:3653:x1871938510594176:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.352656:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29074 count 1 00010000:00010000:1.0:1785220581.352667:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.352679:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cc0f011d/0xf62499ab9278125d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78490 expref: 468 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.352686:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7192:0x0].0x0 since=34s v=(0 0) 00010000:00010000:1.0:1785220581.352689:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7192:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.352692:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.352711:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cc0f011d/0xf62499ab9278125d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78490 expref: 468 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.352721:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cc0f011d/0xf62499ab9278125d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78490 expref: 468 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.352732:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510594944:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.352754:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.352969:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.352977:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.352993:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284058 00000100:00100000:3.0:1785220581.353005:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1064:3655:x1871938510593920:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.353018:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29134 count 1 00010000:00010000:3.0:1785220581.353031:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.353043:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a643087e/0xf62499ab92784929 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f24 expref: 418 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.353049:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ce:0x0].0x0 since=31s v=(0 0) 00010000:00010000:3.0:1785220581.353052:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.353055:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.353073:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a643087e/0xf62499ab92784929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f24 expref: 418 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.353346:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284059 00000100:00100000:0.0:1785220581.353355:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1064:3655:x1871938510593792:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.353369:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29133 count 1 00010000:00010000:0.0:1785220581.353382:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.353395:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ae4f9cfd/0xf62499ab92784842 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ead expref: 418 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.353401:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71cd:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.353403:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.353406:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.353423:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ae4f9cfd/0xf62499ab92784842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ead expref: 418 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.353449:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ae4f9cfd/0xf62499ab92784842 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ead expref: 418 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.353461:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510595200:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.353644:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a643087e/0xf62499ab92784929 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f24 expref: 418 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.353656:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0000_UUID:14210:1871938510595072:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.353724:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.353741:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284060 00000100:00100000:2.0:1785220581.353751:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+987:3655:x1871938510594304:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.353775:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29075 count 1 00010000:00010000:2.0:1785220581.353789:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.353806:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e4314e3e/0xf62499ab927812d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784d6 expref: 468 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.353813:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7193:0x0].0x0 since=34s v=(0 0) 00010000:00010000:2.0:1785220581.353816:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7193:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.353820:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.353840:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e4314e3e/0xf62499ab927812d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784d6 expref: 468 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.353850:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e4314e3e/0xf62499ab927812d4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784d6 expref: 468 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.353860:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0001_UUID:14161:1871938510595328:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.353931:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd480 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.354049:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354059:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354069:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354077:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354113:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.354302:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+988:3654:x1871938510587776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 29031us (29413us total) trans 4295069838 rc 0/0 00000100:00100000:0.0:1785220581.354310:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284034 00000004:00080000:0.0:1785220581.354371:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365c380/2, rc 0, transno 4295069838 00000100:00100000:0.0:1785220581.354382:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510587776:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.354444:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.354462:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904512 00000100:00100000:0.0:1785220581.354467:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904512 00010000:00010000:0.0:1785220581.354491:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000020dfd67b/0xf62499ab92780d2b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7821a expref: 469 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.354516:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99980 00000100:00100000:0.0:1785220581.354524:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9afc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+469:2406:x1871938519904512:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.354533:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.354538:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000020dfd67b/0xf62499ab92780d2b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7821a expref: 469 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354550:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000020dfd67b/0xf62499ab92780d2b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7821a expref: 469 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354574:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.354577:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.354582:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354591:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354606:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.354629:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9afc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+469:2406:x1871938519904512:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.354636:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99980 00000100:00100000:0.0:1785220581.354731:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:13867:1871938510593152:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.354751:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.354814:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510593280:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.354830:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.354859:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.354876:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904768 00000100:00100000:0.0:1785220581.354879:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904768 00010000:00010000:0.0:1785220581.354895:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000042bf09e6/0xf62499ab927844e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d33 expref: 419 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.354917:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99981 00000100:00100000:0.0:1785220581.354924:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+419:2406:x1871938519904768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.354931:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.354936:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000042bf09e6/0xf62499ab927844e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d33 expref: 419 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354947:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000042bf09e6/0xf62499ab927844e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d33 expref: 419 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354962:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042bf09e6/0xf62499ab927844e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d33 expref: 419 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354974:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.354977:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.354981:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.354994:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355010:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.355029:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+418:2406:x1871938519904768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (169us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.355035:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99981 00010000:00010000:0.0:1785220581.355052:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355060:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355069:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355077:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355113:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.355231:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1065:3654:x1871938510592256:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7392us (7447us total) trans 4295070436 rc 0/0 00000100:00100000:0.0:1785220581.355238:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284051 00000004:00080000:0.0:1785220581.355271:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fdc00/2, rc 0, transno 4295070436 00000100:00100000:0.0:1785220581.355282:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510592256:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.355311:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.355328:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904640 00000100:00100000:0.0:1785220581.355331:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904640 00010000:00010000:0.0:1785220581.355347:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006e777c0a/0xf62499ab92784620 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79dbf expref: 418 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.355368:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99982 00000100:00100000:0.0:1785220581.355375:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+418:2406:x1871938519904640:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.355384:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.355389:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006e777c0a/0xf62499ab92784620 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79dbf expref: 418 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355399:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006e777c0a/0xf62499ab92784620 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79dbf expref: 418 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355412:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.355415:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.355419:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355427:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355440:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.355459:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+418:2406:x1871938519904640:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (148us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.355468:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99982 00000100:00100000:0.0:1785220581.355501:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.355515:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519904896 00000100:00100000:0.0:1785220581.355519:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519904896 00010000:00010000:0.0:1785220581.355532:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f126e085/0xf62499ab92780fbd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7832b expref: 469 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.355552:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99983 00000100:00100000:0.0:1785220581.355558:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+469:2406:x1871938519904896:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.355564:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.355568:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f126e085/0xf62499ab92780fbd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7832b expref: 469 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355578:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f126e085/0xf62499ab92780fbd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7832b expref: 469 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355590:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f126e085/0xf62499ab92780fbd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7832b expref: 469 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355600:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.355603:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.355607:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355614:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355630:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.355644:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+468:2406:x1871938519904896:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (143us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.355650:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99983 00010000:00010000:0.0:1785220581.355666:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355675:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355684:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355692:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.355701:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.355804:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+988:3654:x1871938510593024:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7724us (7946us total) trans 4295069839 rc 0/0 00000100:00100000:0.0:1785220581.355810:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284052 00000004:00080000:0.0:1785220581.355837:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955ffb80/2, rc 0, transno 4295069839 00000100:00100000:0.0:1785220581.355842:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510593024:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.356494:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.356503:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356511:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.356515:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356525:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356540:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356548:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4574 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7185:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356575:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.356580:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356586:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.356589:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356598:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356608:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356615:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a45eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356629:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.356632:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356639:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.356642:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356654:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356663:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356670:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a45f2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356685:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.356686:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.356688:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.356718:0:14183:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e777c0a/0xf62499ab92784620 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79dbf expref: 417 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.356729:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0000_UUID:14183:1871938510592896:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.356747:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356757:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356774:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356787:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356793:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.356815:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.356943:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1065:3653:x1871938510592512:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9543us (9986us total) trans 4295070437 rc 0/0 00000100:00100000:1.0:1785220581.356951:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284050 00000100:00100000:1.0:1785220581.356988:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510592640:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.357001:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.357026:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510592768:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.357037:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.357107:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955ff800/2, rc 0, transno 4295070437 00000100:00100000:1.0:1785220581.357114:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510592512:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.357191:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.357204:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510595584:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.357308:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.357340:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510595584 00000100:00100000:1.0:1785220581.357346:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510595584 00000100:00100000:1.0:1785220581.357376:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284061 00000100:00100000:1.0:1785220581.357384:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1065:3653:x1871938510595584:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.357406:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29137 count 1 00010000:00010000:1.0:1785220581.357422:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357433:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000074ae93e2/0xf62499ab92784c7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a09e expref: 416 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357439:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d1:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.357442:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.357445:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357464:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000074ae93e2/0xf62499ab92784c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a09e expref: 416 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357472:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000074ae93e2/0xf62499ab92784c7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a09e expref: 416 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.357480:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0000_UUID:14221:1871938510595712:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.357506:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01d80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.357543:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510595840:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.357559:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.357578:0:14267:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510595840 00000100:00100000:1.0:1785220581.357580:0:14267:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510595840 00000100:00100000:1.0:1785220581.357596:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284062 00000100:00100000:1.0:1785220581.357600:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+988:3653:x1871938510595840:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.357625:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29076 count 1 00010000:00010000:1.0:1785220581.357636:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357644:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b646e878/0xf62499ab92781398 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78538 expref: 467 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357648:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7194:0x0].0x0 since=34s v=(0 0) 00010000:00010000:1.0:1785220581.357650:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7194:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.357652:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357663:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b646e878/0xf62499ab92781398 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78538 expref: 467 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357668:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b646e878/0xf62499ab92781398 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78538 expref: 467 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.357676:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510596096:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.357688:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01900 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.357700:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510595968:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.357713:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.357726:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.357729:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510595456:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.357740:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.357794:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.357818:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905024 00000100:00100000:1.0:1785220581.357822:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905024 00010000:00010000:1.0:1785220581.357840:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000646eab87/0xf62499ab92780df6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78283 expref: 468 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.357863:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99984 00000100:00100000:1.0:1785220581.357869:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+468:2406:x1871938519905024:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.357878:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.357882:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000646eab87/0xf62499ab92780df6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78283 expref: 468 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357894:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000646eab87/0xf62499ab92780df6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78283 expref: 468 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357911:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000646eab87/0xf62499ab92780df6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78283 expref: 468 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357923:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.357926:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x718e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.357930:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357943:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.357960:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.357975:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+467:2406:x1871938519905024:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (181us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.357981:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99984 00000100:00100000:3.0:1785220581.357983:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.357997:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510595968 00010000:00010000:1.0:1785220581.357998:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.358000:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510595968 00010000:00010000:1.0:1785220581.358006:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.358014:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.358021:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510595456 00010000:00010000:1.0:1785220581.358022:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.358023:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510595456 00010000:00010000:1.0:1785220581.358031:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.358037:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284063 00000100:00100000:3.0:1785220581.358042:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+989:3654:x1871938510595968:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.358053:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29077 count 1 00010000:00010000:3.0:1785220581.358065:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358075:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006e466831/0xf62499ab92781447 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78593 expref: 466 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358081:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7195:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.358113:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7195:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.358117:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358147:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006e466831/0xf62499ab92781447 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78593 expref: 466 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.358154:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+990:3653:x1871938510592384:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11103us (11331us total) trans 4295069840 rc 0/0 00010000:00010000:3.0:1785220581.358157:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006e466831/0xf62499ab92781447 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78593 expref: 466 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.358160:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284048 00000100:00100000:3.0:1785220581.358168:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510596224:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.358175:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284064 00000100:00100000:1.0:1785220581.358180:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1066:3654:x1871938510595456:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.358187:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00002000:00080000:1.0:1785220581.358191:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29136 count 1 00010000:00010000:1.0:1785220581.358202:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358213:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.358219:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358232:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.358235:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358243:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358255:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358262:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a45e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358278:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.358282:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358288:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.358291:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358299:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358308:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358315:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a45d6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.358329:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.358329:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.358331:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.358333:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.358355:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003f05ca4d/0xf62499ab92784b21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a004 expref: 416 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.358365:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d0:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.358368:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.358371:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.358391:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003f05ca4d/0xf62499ab92784b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a004 expref: 416 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.358401:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003f05ca4d/0xf62499ab92784b21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a004 expref: 416 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.358412:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510596352:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.358433:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01780 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.358463:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fd500/2, rc 0, transno 4295069840 00000100:00100000:1.0:1785220581.358507:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510592384:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.358545:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.358583:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777b40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.358605:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.358829:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.359534:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905152 00000100:00100000:3.0:1785220581.359540:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905152 00010000:00010000:3.0:1785220581.359568:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006ef89ed9/0xf62499ab927843f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79cbc expref: 417 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.359603:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905408 00000100:00100000:3.0:1785220581.359606:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905408 00010000:00010000:3.0:1785220581.359619:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b3c649a6/0xf62499ab9278479a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e36 expref: 418 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.359637:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905280 00000100:00100000:3.0:1785220581.359640:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905280 00010000:00010000:3.0:1785220581.359654:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000097ca6708/0xf62499ab927810ea lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783b7 expref: 467 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.359675:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99985 00000100:00100000:3.0:1785220581.359682:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+418:2407:x1871938519905152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.359694:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.359699:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006ef89ed9/0xf62499ab927843f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79cbc expref: 418 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359714:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006ef89ed9/0xf62499ab927843f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79cbc expref: 418 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359732:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ef89ed9/0xf62499ab927843f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79cbc expref: 418 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359743:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.359747:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.359752:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359760:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359793:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.359830:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+417:2407:x1871938519905152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 148us (1284us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.359904:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99985 00000100:00100000:3.0:1785220581.359925:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99986 00000100:00100000:3.0:1785220581.359932:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+416:2404:x1871938519905408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.359941:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.359946:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b3c649a6/0xf62499ab9278479a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e36 expref: 416 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359961:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b3c649a6/0xf62499ab9278479a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e36 expref: 416 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359974:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.359977:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.359982:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.359990:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.360003:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.360022:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+416:2404:x1871938519905408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (1415us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.360028:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99986 00000100:00100000:3.0:1785220581.360042:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99987 00000100:00100000:3.0:1785220581.360050:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e10850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+467:2404:x1871938519905280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.360057:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.360063:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000097ca6708/0xf62499ab927810ea lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783b7 expref: 467 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.360072:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000097ca6708/0xf62499ab927810ea lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783b7 expref: 467 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.360113:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7190:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.360116:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7190:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.360120:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.360128:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.360210:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.360307:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e10850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+467:2404:x1871938519905280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 257us (1477us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.360314:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99987 00010000:00010000:1.0:1785220581.360419:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360429:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360439:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360447:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360466:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.360617:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1067:3654:x1871938510592128:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13369us (13739us total) trans 4295070438 rc 0/0 00000100:00100000:1.0:1785220581.360625:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284049 00010000:00010000:1.0:1785220581.360648:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.360655:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360664:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.360669:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360678:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360695:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360702:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a45dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.360721:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.360723:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.360725:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.360757:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fce00/2, rc 0, transno 4295070438 00000100:00100000:1.0:1785220581.360781:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510592128:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.361443:0:14186:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b3c649a6/0xf62499ab9278479a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e36 expref: 415 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.361464:0:14186:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0000_UUID:14186:1871938510594560:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.361483:0:14186:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.361494:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.361502:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.361510:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.361518:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.361533:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.361700:0:14160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000edcca853/0xf62499ab927840c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b26 expref: 414 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.361713:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510590976:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.361730:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.361739:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.361745:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.361759:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.361777:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.361791:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.361888:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d935a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1068:3653:x1871938510590848:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27020us (27182us total) trans 4295070440 rc 0/0 00000100:00100000:3.0:1785220581.361897:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284043 00010000:00010000:3.0:1785220581.361990:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.361996:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362004:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.362008:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362016:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362031:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362039:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4600 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362056:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.362059:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362066:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.362070:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362077:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362105:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362112:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a45b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.362128:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.362130:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.362144:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.362152:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.362367:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1067:3654:x1871938510593408:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10332us (11456us total) trans 4295070439 rc 0/0 00000100:00100000:0.0:1785220581.362375:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284054 00010000:00010000:0.0:1785220581.362413:0:14150:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000097ca6708/0xf62499ab927810ea lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b783b7 expref: 466 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.362425:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510594432:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.362438:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362446:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362452:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362460:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362467:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362487:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.362578:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+990:3653:x1871938510593536:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10751us (11759us total) trans 4295069841 rc 0/0 00000100:00100000:0.0:1785220581.362584:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284053 00010000:00010000:0.0:1785220581.362607:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.362611:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362619:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.362621:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362632:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362644:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362651:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a45f9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7190:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.362666:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.362668:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.362669:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.362758:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510594688:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.362782:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.362872:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510594816:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.362885:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.363019:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510595200:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.363031:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.363139:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.363163:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fca80/2, rc 0, transno 4295070439 00000100:00100000:0.0:1785220581.363169:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510593408:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.363176:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220581.363189:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510596480:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.363420:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.363704:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510596480 00000100:00100000:2.0:1785220581.363710:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510596480 00000100:00100000:2.0:1785220581.363749:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284065 00000100:00100000:2.0:1785220581.363758:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1067:3654:x1871938510596480:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.363794:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29138 count 1 00010000:00010000:2.0:1785220581.363816:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.363832:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f7694559/0xf62499ab92784d2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0e4 expref: 413 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.363841:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d2:0x0].0x0 since=31s v=(0 0) 00010000:00010000:3.0:1785220581.363842:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.363846:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.363851:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.363880:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f7694559/0xf62499ab92784d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0e4 expref: 413 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.363894:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f7694559/0xf62499ab92784d2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0e4 expref: 413 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.363906:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:13837:1871938510596736:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.363957:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.364004:0:14178:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000020dfd67b/0xf62499ab92780d2b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7821a expref: 465 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.364017:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510592000:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.364033:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.364041:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.364054:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.364064:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.364072:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.364166:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.364364:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0000_UUID:14210:1871938510595072:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.364381:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.364535:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510596864:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.365327:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+990:3653:x1871938510591872:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21613us (22313us total) trans 4295069842 rc 0/0 00000100:00100000:2.0:1785220581.365337:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284047 00010000:00010000:3.0:1785220581.365427:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.365436:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.365445:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.365449:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.365460:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.365479:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.365487:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a45cf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x718d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.365505:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.365507:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.365509:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.365685:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.365706:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906688 00000100:00100000:0.0:1785220581.365712:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906688 00010000:00010000:0.0:1785220581.365738:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b646e878/0xf62499ab92781398 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78538 expref: 465 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.365778:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99988 00000100:00100000:0.0:1785220581.365785:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+465:2407:x1871938519906688:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.365798:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.365803:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b646e878/0xf62499ab92781398 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78538 expref: 465 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.365815:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b646e878/0xf62499ab92781398 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78538 expref: 465 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.365834:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7194:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.365904:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7194:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.365912:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.365921:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.365941:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.365977:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+465:2407:x1871938519906688:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 191us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.365984:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99988 00000100:00100000:0.0:1785220581.366020:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.366039:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905536 00000100:00100000:0.0:1785220581.366042:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905536 00010000:00010000:0.0:1785220581.366056:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000baf8d157/0xf62499ab927811bc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7842e expref: 465 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.366077:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99989 00000100:00100000:0.0:1785220581.366083:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+465:2405:x1871938519905536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.366110:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.366115:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000baf8d157/0xf62499ab927811bc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7842e expref: 465 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366126:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000baf8d157/0xf62499ab927811bc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7842e expref: 465 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366151:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000baf8d157/0xf62499ab927811bc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7842e expref: 465 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366167:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7191:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.366170:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7191:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.366175:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366182:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366200:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.366348:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+464:2405:x1871938519905536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 264us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.366355:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99989 00010000:00010000:0.0:1785220581.366376:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366385:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366396:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366403:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366420:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.366539:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+990:3653:x1871938510593664:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14336us (15562us total) trans 4295069843 rc 0/0 00000100:00100000:0.0:1785220581.366546:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284055 00010000:00010000:0.0:1785220581.366567:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.366572:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366580:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.366584:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366593:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366604:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366612:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4607 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7191:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366627:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.366629:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.366631:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.366682:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.366701:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905792 00000100:00100000:0.0:1785220581.366705:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905792 00010000:00010000:0.0:1785220581.366721:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cc0f011d/0xf62499ab9278125d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78490 expref: 464 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.366749:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99990 00000100:00100000:0.0:1785220581.366757:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+464:2405:x1871938519905792:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.366780:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.366785:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cc0f011d/0xf62499ab9278125d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78490 expref: 464 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366799:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cc0f011d/0xf62499ab9278125d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78490 expref: 464 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366811:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7192:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.366814:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7192:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.366818:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.366825:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.366963:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.367011:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b109500/2, rc 0, transno 4295069843 00000100:00100000:1.0:1785220581.367023:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b109500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510593664:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.367036:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa0700/2, rc 0, transno 4295069841 00000100:00100000:1.0:1785220581.367039:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510593536:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.367047:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa1180/2, rc 0, transno 4295069842 00000100:00100000:1.0:1785220581.367050:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510591872:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.367061:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99919c00/2, rc 0, transno 4295070440 00000100:00100000:1.0:1785220581.367069:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510590848:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.367081:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.367103:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510596608:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.367143:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.367185:0:14227:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc0f011d/0xf62499ab9278125d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78490 expref: 464 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.367197:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510594944:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.367351:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.367376:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+463:2405:x1871938519905792:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 619us (694us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.367383:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99990 00000100:00100000:0.0:1785220581.367404:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510596864 00000100:00100000:0.0:1785220581.367408:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510596864 00000100:00100000:0.0:1785220581.367428:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510596608 00000100:00100000:0.0:1785220581.367431:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510596608 00000100:00100000:0.0:1785220581.367447:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284066 00000100:00100000:0.0:1785220581.367452:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1068:3653:x1871938510596864:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.367474:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29139 count 1 00010000:00010000:0.0:1785220581.367492:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367507:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002176ff44/0xf62499ab92784e0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a154 expref: 413 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367514:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d3:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.367517:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.367521:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367546:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002176ff44/0xf62499ab92784e0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a154 expref: 413 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367558:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002176ff44/0xf62499ab92784e0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a154 expref: 413 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.367570:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0000_UUID:14200:1871938510596992:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.367608:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d87b9b300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.367623:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284067 00000100:00100000:0.0:1785220581.367631:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+990:3653:x1871938510596608:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.367644:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29078 count 1 00010000:00010000:0.0:1785220581.367657:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367667:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000918816d9/0xf62499ab9278150b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b785f5 expref: 462 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367673:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7196:0x0].0x0 since=34s v=(0 0) 00010000:00010000:0.0:1785220581.367675:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7196:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.367678:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367696:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000918816d9/0xf62499ab9278150b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b785f5 expref: 462 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.367704:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000918816d9/0xf62499ab9278150b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b785f5 expref: 462 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.367713:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510597120:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.367735:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59d80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.367796:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.367826:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.367907:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.367918:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.367925:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.367934:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.367942:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.367959:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.368044:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0001_UUID:14161:1871938510595328:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.368063:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.368069:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+991:3653:x1871938510594176:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15430us (16616us total) trans 4295069844 rc 0/0 00000100:00100000:1.0:1785220581.368077:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284057 00000100:00100000:1.0:1785220581.368297:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0000_UUID:14221:1871938510595712:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.368313:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.368338:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510596352:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.368349:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368372:0:14267:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b646e878/0xf62499ab92781398 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78538 expref: 462 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.368384:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510596096:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.368394:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368402:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368410:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368418:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368427:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368448:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.368521:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+991:3653:x1871938510595840:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10918us (10959us total) trans 4295069845 rc 0/0 00000100:00100000:1.0:1785220581.368528:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284062 00000100:00100000:1.0:1785220581.368552:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906048 00000100:00100000:1.0:1785220581.368557:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906048 00010000:00010000:1.0:1785220581.368576:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000074ae93e2/0xf62499ab92784c7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a09e expref: 414 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.368598:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906304 00000100:00100000:1.0:1785220581.368601:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906304 00010000:00010000:1.0:1785220581.368612:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006e466831/0xf62499ab92781447 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78593 expref: 462 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.368633:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99991 00000100:00100000:1.0:1785220581.368639:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+414:2405:x1871938519906048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.368649:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.368654:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000074ae93e2/0xf62499ab92784c7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a09e expref: 414 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368665:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000074ae93e2/0xf62499ab92784c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a09e expref: 414 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368685:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074ae93e2/0xf62499ab92784c7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a09e expref: 414 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368700:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.368704:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.368709:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368719:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368736:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.368758:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+413:2405:x1871938519906048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 119us (962us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.368778:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99991 00000100:00100000:1.0:1785220581.368791:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99992 00000100:00100000:1.0:1785220581.368798:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+462:2405:x1871938519906304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.368807:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.368811:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006e466831/0xf62499ab92781447 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78593 expref: 462 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.368822:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006e466831/0xf62499ab92781447 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78593 expref: 462 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.368964:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e466831/0xf62499ab92781447 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78593 expref: 462 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.368975:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510596224:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.369074:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.369123:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.369305:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.369325:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906560 00000100:00100000:0.0:1785220581.369329:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906560 00010000:00010000:0.0:1785220581.369347:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e4314e3e/0xf62499ab927812d4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784d6 expref: 462 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.369367:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905664 00000100:00100000:0.0:1785220581.369370:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905664 00010000:00010000:0.0:1785220581.369382:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003d200ee0/0xf62499ab92784a87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f9b expref: 413 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.369399:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519905920 00000100:00100000:0.0:1785220581.369401:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519905920 00010000:00010000:0.0:1785220581.369413:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ae4f9cfd/0xf62499ab92784842 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ead expref: 414 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.369431:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99993 00000100:00100000:0.0:1785220581.369438:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+462:2405:x1871938519906560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.369446:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.369450:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e4314e3e/0xf62499ab927812d4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784d6 expref: 462 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369461:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e4314e3e/0xf62499ab927812d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784d6 expref: 462 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369475:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e4314e3e/0xf62499ab927812d4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b784d6 expref: 462 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369486:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7193:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.369489:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7193:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.369494:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369503:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369520:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.369541:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+461:2405:x1871938519906560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (467us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.369549:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99993 00000100:00100000:0.0:1785220581.369562:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99994 00000100:00100000:0.0:1785220581.369569:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac547450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+414:2405:x1871938519905664:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.369576:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.369581:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003d200ee0/0xf62499ab92784a87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f9b expref: 414 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369591:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003d200ee0/0xf62499ab92784a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f9b expref: 414 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369604:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d200ee0/0xf62499ab92784a87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f9b expref: 414 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369614:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.369616:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.369621:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369634:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369650:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.369667:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac547450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+413:2405:x1871938519905664:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (543us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.369673:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99994 00000100:00100000:0.0:1785220581.369686:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99995 00000100:00100000:0.0:1785220581.369693:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2405:x1871938519905920:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.369700:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.369704:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ae4f9cfd/0xf62499ab92784842 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ead expref: 412 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369714:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ae4f9cfd/0xf62499ab92784842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ead expref: 412 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369727:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae4f9cfd/0xf62499ab92784842 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ead expref: 412 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369737:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.369740:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.369744:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369757:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369787:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.369805:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+411:2405:x1871938519905920:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (500us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.369811:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99995 00010000:00010000:0.0:1785220581.369832:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369847:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369857:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369866:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.369880:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.369986:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1069:3653:x1871938510594048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17561us (18601us total) trans 4295070441 rc 0/0 00000100:00100000:0.0:1785220581.369994:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284056 00010000:00010000:0.0:1785220581.370013:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370027:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370036:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370044:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370057:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.370148:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1069:3655:x1871938510593792:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16794us (18605us total) trans 4295070442 rc 0/0 00000100:00100000:0.0:1785220581.370155:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284059 00010000:00010000:0.0:1785220581.370178:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220581.370185:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370194:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.370198:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370208:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.370452:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7195:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.370457:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7195:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.370462:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.370473:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370511:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370520:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4615 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7192:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370536:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.370540:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370547:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.370550:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370558:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370567:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370575:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4638 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7194:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370588:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.370592:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370598:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.370602:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370610:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370619:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370626:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a460e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370639:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.370643:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370650:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.370653:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370660:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370670:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370676:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a4623 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.370690:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.370692:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.370694:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.370727:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10b100/2, rc 0, transno 4295070442 00000100:00100000:0.0:1785220581.370733:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510593792:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.370971:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.370995:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906176 00000100:00100000:0.0:1785220581.371000:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906176 00010000:00010000:0.0:1785220581.371019:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a643087e/0xf62499ab92784929 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f24 expref: 411 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.371045:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99996 00000100:00100000:0.0:1785220581.371052:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+411:2405:x1871938519906176:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.371062:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.371068:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a643087e/0xf62499ab92784929 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f24 expref: 411 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.371079:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a643087e/0xf62499ab92784929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f24 expref: 411 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.371110:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a643087e/0xf62499ab92784929 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f24 expref: 411 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.371120:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.371123:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.371127:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.371147:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.371153:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.371163:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.371165:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.371171:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.371178:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.371188:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+410:2405:x1871938519906176:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 137us (219us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.371192:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.371195:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99996 00010000:00010000:3.0:1785220581.371436:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.371456:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.371464:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.371471:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.371477:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.371485:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220581.371597:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.371620:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+460:2405:x1871938519906304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2822us (3794us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.371627:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99992 00000004:00080000:1.0:1785220581.371691:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365c700/2, rc 0, transno 4295069845 00000100:00100000:1.0:1785220581.371697:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510595840:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.371707:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b109f80/2, rc 0, transno 4295069844 00000100:00100000:1.0:1785220581.371709:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b109f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510594176:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.371718:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10ad80/2, rc 0, transno 4295070441 00000100:00100000:1.0:1785220581.371722:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510594048:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.371742:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371750:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371757:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371774:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371788:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220581.371883:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1070:3653:x1871938510595584:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14499us (14575us total) trans 4295070444 rc 0/0 00000100:00100000:1.0:1785220581.371890:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284061 00000004:00080000:1.0:1785220581.371916:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b108000/2, rc 0, transno 4295070444 00000100:00100000:1.0:1785220581.371921:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510595584:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.371933:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.371938:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371946:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.371950:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371958:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371971:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371978:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a462a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7193:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.371994:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.371998:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372004:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.372007:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372015:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372024:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372030:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a461c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372044:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.372047:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372054:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.372057:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372064:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372077:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372098:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a77a8686/0xf62499ab927a4631 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.372111:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.372113:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.372115:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.372121:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.374857:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+991:3655:x1871938510594304:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21098us (23275us total) trans 4295069846 rc 0/0 00000100:00100000:2.0:1785220581.374874:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284060 00000100:00100000:0.0:1785220581.375033:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510597376:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.375107:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.375296:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b108380/2, rc 0, transno 4295069846 00000100:00100000:0.0:1785220581.375304:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510594304:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.375370:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9365d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510597632:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.375396:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.375423:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.375429:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510597504:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.375449:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.375570:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1069:3655:x1871938510593920:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22567us (24065us total) trans 4295070443 rc 0/0 00000100:00100000:3.0:1785220581.375578:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284058 00000004:00080000:0.0:1785220581.375616:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10b800/2, rc 0, transno 4295070443 00000100:00100000:0.0:1785220581.375627:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510593920:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.375636:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.375640:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510597248:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.377446:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97843b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510597760:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.377496:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.377656:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510597376 00000100:00100000:3.0:1785220581.377663:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510597376 00000100:00100000:0.0:1785220581.377756:0:14203:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510597632 00000100:00100000:0.0:1785220581.377761:0:14203:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510597632 00000100:00100000:0.0:1785220581.377797:0:14203:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510597504 00000100:00100000:0.0:1785220581.377800:0:14203:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510597504 00000100:00100000:0.0:1785220581.377820:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284068 00000100:00100000:0.0:1785220581.377829:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1071:3655:x1871938510597376:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.377914:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29141 count 1 00010000:00010000:0.0:1785220581.377941:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.377963:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007bfdfd21/0xf62499ab92784fe3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a218 expref: 409 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.377973:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d5:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.377977:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.377982:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378010:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007bfdfd21/0xf62499ab92784fe3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a218 expref: 409 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378022:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007bfdfd21/0xf62499ab92784fe3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a218 expref: 409 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.378035:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510597888:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.378170:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.378325:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.378351:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906432 00000100:00100000:0.0:1785220581.378355:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906432 00010000:00010000:0.0:1785220581.378376:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003f05ca4d/0xf62499ab92784b21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a004 expref: 410 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.378401:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99997 00000100:00100000:0.0:1785220581.378409:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+410:2405:x1871938519906432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.378418:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.378423:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003f05ca4d/0xf62499ab92784b21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a004 expref: 410 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378433:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003f05ca4d/0xf62499ab92784b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a004 expref: 410 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378452:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f05ca4d/0xf62499ab92784b21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a004 expref: 410 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378463:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.378466:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.378471:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378480:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378500:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.378522:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+409:2405:x1871938519906432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (198us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.378529:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99997 00000100:00100000:0.0:1785220581.378603:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.378617:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906816 00000100:00100000:0.0:1785220581.378620:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906816 00010000:00010000:0.0:1785220581.378633:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f7694559/0xf62499ab92784d2e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0e4 expref: 409 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.378653:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99998 00000100:00100000:0.0:1785220581.378658:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+409:2406:x1871938519906816:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.378665:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.378669:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f7694559/0xf62499ab92784d2e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0e4 expref: 409 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.378670:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284069 00000100:00100000:2.0:1785220581.378679:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1071:3653:x1871938510597632:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.378679:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f7694559/0xf62499ab92784d2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0e4 expref: 409 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378691:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d2:0x0].0x0 since=0s v=(1 0) 00002000:00080000:2.0:1785220581.378692:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29143 count 1 00010000:00010000:0.0:1785220581.378695:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.378698:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.378704:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378706:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.378715:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000097edf8ee/0xf62499ab927851b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a2ff expref: 409 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.378717:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.378723:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d7:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.378725:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.378729:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.378733:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+409:2406:x1871938519906816:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (130us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.378739:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99998 00010000:00010000:2.0:1785220581.378747:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000097edf8ee/0xf62499ab927851b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a2ff expref: 408 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.378820:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510597120:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.378903:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000097edf8ee/0xf62499ab927851b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a2ff expref: 408 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.378917:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510598016:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.378950:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.378950:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.378965:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510597760 00000100:00100000:2.0:1785220581.378969:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510597760 00000100:00100000:0.0:1785220581.378974:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284070 00000100:00100000:0.0:1785220581.378980:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1072:3653:x1871938510597504:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.378993:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29142 count 1 00000100:00100000:2.0:1785220581.378994:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284071 00000100:00100000:2.0:1785220581.379000:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1072:3654:x1871938510597760:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.379006:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.379008:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29144 count 1 00010000:00010000:0.0:1785220581.379018:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ba477811/0xf62499ab927850a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a281 expref: 408 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.379019:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.379026:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d6:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.379029:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.379030:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eb8ad840/0xf62499ab92785260 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a361 expref: 408 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.379033:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.379037:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d8:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.379040:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.379043:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.379049:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ba477811/0xf62499ab927850a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a281 expref: 408 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.379056:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eb8ad840/0xf62499ab92785260 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a361 expref: 408 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.379058:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ba477811/0xf62499ab927850a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a281 expref: 408 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.379066:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eb8ad840/0xf62499ab92785260 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a361 expref: 408 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.379069:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510598144:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.379081:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510598272:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.379303:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.379312:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.379321:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.379329:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.379348:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.379562:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379571:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379578:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379590:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379601:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.379684:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1073:3654:x1871938510595456:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21504us (21942us total) trans 4295070445 rc 0/0 00000100:00100000:1.0:1785220581.379691:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284064 00010000:00010000:1.0:1785220581.379709:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.379714:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379722:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.379726:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379735:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379747:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379754:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000142c7479/0xf62499ab927a463f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7195:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379789:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.379793:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379799:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.379801:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379808:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379817:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379823:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4646 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.379892:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.379894:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.379896:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.379901:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.379923:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10b480/2, rc 0, transno 4295070445 00000100:00100000:1.0:1785220581.379935:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510595456:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.379995:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.380001:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510598400:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.380021:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.380039:0:14261:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510598400 00000100:00100000:1.0:1785220581.380042:0:14261:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510598400 00000100:00100000:1.0:1785220581.380062:0:14261:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284072 00000100:00100000:1.0:1785220581.380067:0:14261:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1073:3654:x1871938510598400:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.380078:0:14261:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29145 count 1 00010000:00010000:1.0:1785220581.380107:0:14261:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.380122:0:14261:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000096d0a813/0xf62499ab9278532b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3ca expref: 408 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.380129:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d9:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.380132:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.380136:0:14261:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.380158:0:14261:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000096d0a813/0xf62499ab9278532b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3ca expref: 408 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.380169:0:14261:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000096d0a813/0xf62499ab9278532b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3ca expref: 408 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.380180:0:14261:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0000_UUID:14261:1871938510598528:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.380291:0:14261:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01d80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.380351:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.380409:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.380416:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510598656:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.380441:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.380475:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510598784:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.380492:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.380537:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.380541:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510599168:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.380558:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.380576:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510598912:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.380591:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.380603:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.380607:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510599040:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.380620:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.380650:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510599296:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.380666:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.380751:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777b40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.380751:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59d80 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.380796:0:13837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f7694559/0xf62499ab92784d2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0e4 expref: 408 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.380811:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:13837:1871938510596736:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.380822:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.380829:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.380838:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.380846:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.380855:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.380862:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.380878:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.380984:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0000_UUID:14200:1871938510596992:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.380989:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1074:3654:x1871938510596480:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17233us (17570us total) trans 4295070446 rc 0/0 00000100:00100000:2.0:1785220581.380997:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284065 00010000:00010000:0.0:1785220581.381001:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.381013:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510597248 00000100:00100000:2.0:1785220581.381016:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510597248 00000100:00100000:2.0:1785220581.381037:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510598656 00000100:00100000:2.0:1785220581.381040:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510598656 00000100:00100000:2.0:1785220581.381055:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510598784 00000100:00100000:2.0:1785220581.381057:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510598784 00000100:00100000:2.0:1785220581.381068:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510599168 00000100:00100000:2.0:1785220581.381071:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510599168 00000100:00100000:0.0:1785220581.381296:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.381402:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+994:3654:x1871938510595968:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23358us (23686us total) trans 4295069847 rc 0/0 00000100:00100000:3.0:1785220581.381410:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284063 00000100:00100000:3.0:1785220581.381425:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510599040 00000100:00100000:3.0:1785220581.381428:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510599040 00000100:00100000:3.0:1785220581.381450:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510599296 00000100:00100000:3.0:1785220581.381453:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510599296 00000100:00100000:3.0:1785220581.381470:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284073 00000100:00100000:3.0:1785220581.381478:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8e00fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1074:3655:x1871938510597248:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.381485:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510598912 00000100:00100000:2.0:1785220581.381489:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510598912 00002000:00080000:3.0:1785220581.381495:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29140 count 1 00010000:00010000:3.0:1785220581.381511:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.381511:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284074 00000100:00100000:2.0:1785220581.381517:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+996:3653:x1871938510598656:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.381525:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009dbd57b8/0xf62499ab92784f11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1af expref: 407 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.381527:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29079 count 1 00010000:00010000:3.0:1785220581.381533:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71d4:0x0].0x0 since=31s v=(0 0) 00010000:00010000:3.0:1785220581.381536:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.381538:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381540:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381550:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b6b18d6a/0xf62499ab927815dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78665 expref: 459 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381556:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7197:0x0].0x0 since=34s v=(0 0) 00010000:00010000:2.0:1785220581.381558:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7197:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.381562:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381563:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009dbd57b8/0xf62499ab92784f11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1af expref: 407 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381574:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009dbd57b8/0xf62499ab92784f11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1af expref: 407 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381579:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b6b18d6a/0xf62499ab927815dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78665 expref: 459 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.381587:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510599424:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.381593:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b6b18d6a/0xf62499ab927815dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78665 expref: 459 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.381602:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510599552:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.381619:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.381619:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.381637:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284075 00000100:00100000:2.0:1785220581.381641:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284076 00000100:00100000:3.0:1785220581.381644:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+996:3653:x1871938510598784:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.381647:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+996:3653:x1871938510599168:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.381656:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29080 count 1 00002000:00080000:2.0:1785220581.381656:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29083 count 1 00010000:00010000:3.0:1785220581.381667:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381667:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381680:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c89b6a55/0xf62499ab927816bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b786d5 expref: 459 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381680:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002d64f36d/0xf62499ab92781941 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78833 expref: 459 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381688:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7198:0x0].0x0 since=34s v=(0 0) 00010000:00010000:2.0:1785220581.381688:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x719b:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.381691:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7198:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.381691:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.381696:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381696:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381713:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c89b6a55/0xf62499ab927816bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b786d5 expref: 459 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381713:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002d64f36d/0xf62499ab92781941 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78833 expref: 459 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381724:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c89b6a55/0xf62499ab927816bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b786d5 expref: 459 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381724:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002d64f36d/0xf62499ab92781941 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78833 expref: 459 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.381741:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510599680:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.381741:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510599808:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.381779:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9ae74c00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.381795:0:14175:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284077 00000100:00100000:2.0:1785220581.381802:0:14175:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+996:3654:x1871938510599040:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.381813:0:14175:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29082 count 1 00000100:00100000:3.0:1785220581.381821:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.381825:0:14175:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381836:0:14175:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b76d5682/0xf62499ab9278186f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787bc expref: 459 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381844:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x719a:0x0].0x0 since=34s v=(0 0) 00010000:00010000:2.0:1785220581.381846:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.381850:0:14175:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381869:0:14175:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b76d5682/0xf62499ab9278186f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787bc expref: 459 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381879:0:14175:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b76d5682/0xf62499ab9278186f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787bc expref: 459 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.381889:0:14175:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510599936:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.381895:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284078 00000100:00100000:3.0:1785220581.381903:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+996:3653:x1871938510599296:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.381910:0:14175:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920c00 going to sleep for 11 seconds 00002000:00080000:3.0:1785220581.381915:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29084 count 1 00010000:00010000:3.0:1785220581.381925:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.381926:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284079 00000100:00100000:2.0:1785220581.381932:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+996:3654:x1871938510598912:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.381937:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000063e83bb9/0xf62499ab92781a1a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788aa expref: 459 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.381941:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29081 count 1 00010000:00010000:3.0:1785220581.381944:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x719c:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.381947:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.381951:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381951:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381960:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000056b74ff9/0xf62499ab9278176c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78737 expref: 459 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381967:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7199:0x0].0x0 since=34s v=(0 0) 00010000:00010000:3.0:1785220581.381969:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000063e83bb9/0xf62499ab92781a1a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788aa expref: 459 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381970:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7199:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.381974:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.381978:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000063e83bb9/0xf62499ab92781a1a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788aa expref: 459 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.381988:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510600064:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.381989:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000056b74ff9/0xf62499ab9278176c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78737 expref: 459 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.381998:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000056b74ff9/0xf62499ab9278176c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78737 expref: 459 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.382007:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.382007:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510600192:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.382022:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.382039:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.382045:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.382053:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.382056:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.382057:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.382066:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.382069:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.382082:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.382118:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a464d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.382136:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.382138:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.382140:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.382334:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519906944 00000100:00100000:1.0:1785220581.382338:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519906944 00010000:00010000:1.0:1785220581.382360:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000918816d9/0xf62499ab9278150b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b785f5 expref: 460 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.382390:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907072 00000100:00100000:1.0:1785220581.382393:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907072 00010000:00010000:1.0:1785220581.382405:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002176ff44/0xf62499ab92784e0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a154 expref: 408 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.382428:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 99999 00000100:00100000:1.0:1785220581.382435:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+460:2407:x1871938519906944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.382445:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.382447:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100000 00010000:00010000:1.0:1785220581.382450:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000918816d9/0xf62499ab9278150b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b785f5 expref: 460 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.382454:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+408:2407:x1871938519907072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.382462:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.382462:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000918816d9/0xf62499ab9278150b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b785f5 expref: 460 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.382466:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002176ff44/0xf62499ab92784e0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a154 expref: 408 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382478:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000918816d9/0xf62499ab9278150b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b785f5 expref: 460 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382487:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7196:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.382490:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7196:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.382495:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382502:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382520:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.382537:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+459:2407:x1871938519906944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (1715us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.382543:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 99999 00010000:00010000:1.0:1785220581.382565:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382573:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382582:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382589:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382603:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.382726:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+997:3653:x1871938510596608:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15096us (15593us total) trans 4295069848 rc 0/0 00000100:00100000:1.0:1785220581.382733:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284067 00000004:00080000:1.0:1785220581.382780:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93665f80/2, rc 0, transno 4295069848 00000100:00100000:1.0:1785220581.382791:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510596608:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.382806:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.382810:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382818:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.382822:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382829:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.382837:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002176ff44/0xf62499ab92784e0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a154 expref: 408 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.382852:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002176ff44/0xf62499ab92784e0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a154 expref: 408 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.382862:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.382864:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.382869:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.382876:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.382896:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.382912:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+407:2407:x1871938519907072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 458us (1617us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.382918:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100000 00010000:00010000:1.0:1785220581.382948:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382957:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a465b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7196:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.382975:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.382976:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.382978:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.383003:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbf100/2, rc 0, transno 4295070446 00000100:00100000:1.0:1785220581.383013:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510596480:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.383023:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbc700/2, rc 0, transno 4295069847 00000100:00100000:1.0:1785220581.383026:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510595968:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.383075:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6eca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510600320:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.383169:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.383201:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510600320 00000100:00100000:1.0:1785220581.383205:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510600320 00000100:00100000:1.0:1785220581.383309:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284080 00000100:00100000:1.0:1785220581.383317:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1075:3654:x1871938510600320:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.383336:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29146 count 1 00010000:00010000:1.0:1785220581.383351:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.383364:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f43a8766/0xf62499ab92785386 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3fb expref: 406 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.383370:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71da:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.383373:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.383377:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.383401:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f43a8766/0xf62499ab92785386 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3fb expref: 406 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.383413:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f43a8766/0xf62499ab92785386 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3fb expref: 406 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.383424:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510600448:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.383455:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.383545:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.383552:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510600704:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.383574:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.383601:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.383606:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510600576:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.383622:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.383715:0:14205:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510600704 00000100:00100000:1.0:1785220581.383718:0:14205:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510600704 00000100:00100000:1.0:1785220581.384441:0:14205:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510600576 00000100:00100000:1.0:1785220581.384447:0:14205:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510600576 00000100:00100000:3.0:1785220581.384610:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284081 00000100:00100000:3.0:1785220581.384623:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+998:3653:x1871938510600704:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.384646:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29086 count 1 00010000:00010000:3.0:1785220581.384666:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384688:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a1b9cab9/0xf62499ab92781bfd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7899f expref: 458 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384695:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x719e:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.384698:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.384702:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384728:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a1b9cab9/0xf62499ab92781bfd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7899f expref: 458 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384739:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a1b9cab9/0xf62499ab92781bfd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7899f expref: 458 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.384753:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510600832:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.384915:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.384932:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384942:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384954:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384969:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.384988:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.385153:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1076:3653:x1871938510596864:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17699us (18188us total) trans 4295070447 rc 0/0 00000100:00100000:3.0:1785220581.385161:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284066 00000100:00100000:3.0:1785220581.385179:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284082 00000100:00100000:3.0:1785220581.385187:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+998:3654:x1871938510600576:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.385199:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29085 count 1 00010000:00010000:3.0:1785220581.385212:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385303:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007315256f/0xf62499ab92781b16 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7891a expref: 458 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385311:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x719d:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.385314:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.385319:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385342:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007315256f/0xf62499ab92781b16 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7891a expref: 458 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385352:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007315256f/0xf62499ab92781b16 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7891a expref: 458 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.385363:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510600960:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.385389:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.385491:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbe300/2, rc 0, transno 4295070447 00000100:00100000:1.0:1785220581.385503:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510596864:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.385556:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510601088:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.385584:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.385667:0:14193:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510601088 00000100:00100000:3.0:1785220581.385673:0:14193:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510601088 00000100:00100000:3.0:1785220581.385710:0:14193:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284083 00000100:00100000:3.0:1785220581.385718:0:14193:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1076:3653:x1871938510601088:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.385731:0:14193:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29147 count 1 00010000:00010000:3.0:1785220581.385745:0:14193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385758:0:14193:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006448a006/0xf62499ab9278544a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a44f expref: 406 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385776:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71db:0x0].0x0 since=31s v=(0 0) 00010000:00010000:3.0:1785220581.385780:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.385784:0:14193:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385806:0:14193:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006448a006/0xf62499ab9278544a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a44f expref: 406 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.385817:0:14193:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006448a006/0xf62499ab9278544a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a44f expref: 406 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.385905:0:14193:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510601216:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.385935:0:14193:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.389204:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.389220:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389238:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.389243:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389253:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389272:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389280:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a4654 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389301:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.389303:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.389305:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.389425:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.389451:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907328 00000100:00100000:0.0:1785220581.389458:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907328 00010000:00010000:0.0:1785220581.389495:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002d64f36d/0xf62499ab92781941 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78833 expref: 459 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.389528:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100001 00000100:00100000:0.0:1785220581.389538:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+459:2407:x1871938519907328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.389554:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.389560:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002d64f36d/0xf62499ab92781941 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78833 expref: 459 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.389572:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002d64f36d/0xf62499ab92781941 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78833 expref: 459 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.389591:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.389595:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.389600:0:14196:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d64f36d/0xf62499ab92781941 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78833 expref: 459 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.389601:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.389612:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.389614:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510599808:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.389634:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.389634:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.389643:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389650:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389659:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389667:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.389672:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+458:2407:x1871938519907328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 136us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.389679:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100001 00010000:00010000:1.0:1785220581.389682:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.389740:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.389854:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.389863:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+999:3653:x1871938510599168:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8214us (9302us total) trans 4295069849 rc 0/0 00000100:00100000:0.0:1785220581.389869:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.389872:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284076 00000100:00100000:0.0:1785220581.389883:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.389896:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907584 00000100:00100000:0.0:1785220581.389898:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.389900:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907584 00010000:00010000:1.0:1785220581.389915:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000063e83bb9/0xf62499ab92781a1a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788aa expref: 458 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.389935:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907840 00000100:00100000:1.0:1785220581.389938:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907840 00010000:00010000:1.0:1785220581.389951:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f43a8766/0xf62499ab92785386 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3fb expref: 407 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.389958:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510598144:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.389967:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907200 00000100:00100000:1.0:1785220581.389970:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907200 00010000:00010000:0.0:1785220581.389971:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.389980:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c89b6a55/0xf62499ab927816bd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b786d5 expref: 459 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.389986:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907456 00000100:00100000:0.0:1785220581.389988:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907456 00000100:00100000:1.0:1785220581.389997:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907712 00000100:00100000:1.0:1785220581.390000:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907712 00010000:00010000:0.0:1785220581.390001:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b76d5682/0xf62499ab9278186f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787bc expref: 460 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390009:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000056b74ff9/0xf62499ab9278176c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78737 expref: 461 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.390019:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100002 00000100:00100000:1.0:1785220581.390024:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100003 00000100:00100000:0.0:1785220581.390026:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+461:2407:x1871938519907584:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.390030:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+407:2407:x1871938519907840:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.390033:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.390037:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.390037:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000063e83bb9/0xf62499ab92781a1a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788aa expref: 461 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390041:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f43a8766/0xf62499ab92785386 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3fb expref: 407 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390051:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f43a8766/0xf62499ab92785386 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3fb expref: 407 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.390053:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000063e83bb9/0xf62499ab92781a1a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788aa expref: 461 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390065:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.390066:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.390070:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.390070:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.390075:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.390075:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390103:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.390106:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.390117:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.390118:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.390139:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+407:2407:x1871938519907840:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.390146:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100003 00000100:00100000:1.0:1785220581.390161:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100004 00000100:00100000:1.0:1785220581.390168:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+461:2407:x1871938519907200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.390175:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.390180:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c89b6a55/0xf62499ab927816bd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b786d5 expref: 461 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.390187:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510599680:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.390190:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c89b6a55/0xf62499ab927816bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b786d5 expref: 461 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.390203:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390203:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c89b6a55/0xf62499ab927816bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b786d5 expref: 461 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390215:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7198:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.390218:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7198:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.390222:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390229:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.390239:0:14201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000063e83bb9/0xf62499ab92781a1a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b788aa expref: 460 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.390245:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.390249:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510600064:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.390264:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.390272:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.390279:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.390287:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.390294:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.390309:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.390729:0:14175:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510599936:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.390744:0:14175:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.390783:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510600192:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.390795:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.390819:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510598016:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.390830:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390846:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.390852:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390859:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.390862:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390871:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390884:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390891:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4693 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390913:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.390916:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390922:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.390925:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390932:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390942:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390949:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a46a8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.390965:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.390967:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.390969:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.391284:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+459:2407:x1871938519907584:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1189us (1474us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.391294:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100002 00000100:00100000:0.0:1785220581.391317:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100005 00000100:00100000:0.0:1785220581.391326:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+458:2407:x1871938519907456:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.391337:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.391342:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b76d5682/0xf62499ab9278186f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787bc expref: 458 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391352:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b76d5682/0xf62499ab9278186f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787bc expref: 458 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391371:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b76d5682/0xf62499ab9278186f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b787bc expref: 458 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391393:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.391396:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.391400:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391408:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391438:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.391462:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+457:2407:x1871938519907456:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 137us (1577us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.391469:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100005 00000100:00100000:0.0:1785220581.391483:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100006 00000100:00100000:0.0:1785220581.391490:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+456:2407:x1871938519907712:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.391497:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.391501:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000056b74ff9/0xf62499ab9278176c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78737 expref: 456 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391510:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000056b74ff9/0xf62499ab9278176c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78737 expref: 456 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391527:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000056b74ff9/0xf62499ab9278176c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78737 expref: 456 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391539:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7199:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.391542:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7199:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.391546:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391554:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391570:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.391588:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+455:2407:x1871938519907712:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (1689us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.391594:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100006 00000100:00100000:0.0:1785220581.391712:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.391748:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.391793:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.391812:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519907968 00000100:00100000:0.0:1785220581.391818:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519907968 00010000:00010000:0.0:1785220581.391838:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000097edf8ee/0xf62499ab927851b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a2ff expref: 407 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.391861:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908224 00000100:00100000:0.0:1785220581.391864:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908224 00010000:00010000:0.0:1785220581.391877:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ba477811/0xf62499ab927850a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a281 expref: 408 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.391894:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908096 00000100:00100000:1.0:1785220581.391896:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+454:2407:x1871938519907200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1670us (1967us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.391896:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908096 00000100:00100000:1.0:1785220581.391903:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100004 00010000:00010000:0.0:1785220581.391908:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000096d0a813/0xf62499ab9278532b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3ca expref: 409 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.391919:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100007 00000100:00100000:0.0:1785220581.391923:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100008 00000100:00100000:1.0:1785220581.391926:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+409:2406:x1871938519907968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.391927:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9af050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+409:2406:x1871938519908224:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.391935:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.391935:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.391941:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000097edf8ee/0xf62499ab927851b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a2ff expref: 409 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391941:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ba477811/0xf62499ab927850a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a281 expref: 409 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.391953:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000097edf8ee/0xf62499ab927851b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a2ff expref: 409 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391953:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ba477811/0xf62499ab927850a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a281 expref: 409 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.391968:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097edf8ee/0xf62499ab927851b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a2ff expref: 409 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391968:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba477811/0xf62499ab927850a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a281 expref: 409 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.391979:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.391979:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.391983:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.391983:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.391988:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391988:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.391998:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.391998:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392014:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.392014:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.392041:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+407:2406:x1871938519907968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.392048:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100007 00000100:00100000:1.0:1785220581.392062:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100009 00000100:00100000:1.0:1785220581.392068:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+406:2406:x1871938519908096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.392076:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.392080:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000096d0a813/0xf62499ab9278532b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3ca expref: 406 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.392302:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9af050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+406:2406:x1871938519908224:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.392310:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100008 00010000:00010000:0.0:1785220581.392355:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392365:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392376:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392384:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392394:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000096d0a813/0xf62499ab9278532b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3ca expref: 405 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392400:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.392408:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.392410:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.392414:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392421:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392432:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.392450:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+405:2406:x1871938519908096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 384us (658us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.392456:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100009 00010000:00010000:1.0:1785220581.392498:0:14261:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096d0a813/0xf62499ab9278532b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3ca expref: 404 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.392509:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0000_UUID:14261:1871938510598528:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.392525:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.392530:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1077:3653:x1871938510597504:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13549us (17079us total) trans 4295070448 rc 0/0 00000100:00100000:0.0:1785220581.392537:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284070 00010000:00010000:1.0:1785220581.392540:0:14261:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392546:0:14261:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392554:0:14261:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392561:0:14261:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392573:0:14261:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.392592:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.392608:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908352 00000100:00100000:0.0:1785220581.392612:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908352 00010000:00010000:0.0:1785220581.392629:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eb8ad840/0xf62499ab92785260 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a361 expref: 404 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.392654:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100010 00000100:00100000:1.0:1785220581.392655:0:14261:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1077:3654:x1871938510598400:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12587us (12632us total) trans 4295070449 rc 0/0 00000100:00100000:0.0:1785220581.392661:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+404:2404:x1871938519908352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.392662:0:14261:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284072 00010000:00010000:0.0:1785220581.392668:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.392674:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eb8ad840/0xf62499ab92785260 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a361 expref: 404 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392683:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eb8ad840/0xf62499ab92785260 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a361 expref: 404 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392684:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392692:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392695:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.392698:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.392701:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392703:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392711:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.392714:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.392723:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.392726:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.392741:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+404:2404:x1871938519908352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 80us (149us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.392746:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100010 00000100:00100000:1.0:1785220581.392820:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1000:3653:x1871938510598784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11175us (12325us total) trans 4295069851 rc 0/0 00000100:00100000:1.0:1785220581.392826:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284075 00000100:00100000:0.0:1785220581.392847:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.392878:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.392891:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908608 00000100:00100000:0.0:1785220581.392894:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908608 00010000:00010000:0.0:1785220581.392909:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b6b18d6a/0xf62499ab927815dd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78665 expref: 454 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.392926:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908480 00000100:00100000:0.0:1785220581.392929:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908480 00000004:00080000:1.0:1785220581.392941:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ef480/2, rc 0, transno 4295069849 00010000:00010000:0.0:1785220581.392941:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009dbd57b8/0xf62499ab92784f11 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1af expref: 404 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.392952:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510599168:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.392956:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100011 00000004:00080000:1.0:1785220581.392962:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ee300/2, rc 0, transno 4295069851 00000100:00100000:0.0:1785220581.392962:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+454:2406:x1871938519908608:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.392966:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510598784:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.392969:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.392972:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b6b18d6a/0xf62499ab927815dd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78665 expref: 454 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.392975:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbea00/2, rc 0, transno 4295070448 00010000:00010000:0.0:1785220581.392982:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b6b18d6a/0xf62499ab927815dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78665 expref: 454 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.392984:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510597504:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.392995:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7197:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.392999:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7197:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.393001:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100012 00010000:00010000:0.0:1785220581.393003:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.393007:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+404:2406:x1871938519908480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.393012:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393014:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.393019:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009dbd57b8/0xf62499ab92784f11 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1af expref: 404 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.393023:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.393028:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009dbd57b8/0xf62499ab92784f11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1af expref: 404 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.393038:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+454:2406:x1871938519908608:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (192us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.393040:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.393043:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d4:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.393044:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100011 00010000:00010000:1.0:1785220581.393047:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393055:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393066:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.393082:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+404:2406:x1871938519908480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.393259:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100012 00010000:00010000:1.0:1785220581.393290:0:13836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f43a8766/0xf62499ab92785386 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3fb expref: 403 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.393300:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510600448:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.393307:0:14175:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393312:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.393316:0:14175:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393321:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393329:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.393331:0:14175:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393337:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.393338:0:14175:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.393347:0:14175:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:1.0:1785220581.393350:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393357:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1785220581.393426:0:14175:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1000:3654:x1871938510599040:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11623us (12803us total) trans 4295069852 rc 0/0 00000100:00100000:1.0:1785220581.393427:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1077:3654:x1871938510600320:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10109us (10256us total) trans 4295070450 rc 0/0 00000100:00100000:1.0:1785220581.393434:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284080 00010000:00010000:1.0:1785220581.393451:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.393457:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393464:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.393469:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393478:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393491:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393499:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4670 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393516:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.393520:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393525:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.393528:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393535:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393544:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393550:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a467e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393564:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.393567:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393573:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.393576:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393583:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393598:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393605:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a469a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7198:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393619:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.393622:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393628:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fec00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.393631:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393638:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393646:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393653:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a46a1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393665:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.393669:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393675:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.393678:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393684:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393692:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393698:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a46b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.393708:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.393709:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.393711:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.393721:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.393778:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1000:3653:x1871938510599296:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11864us (13098us total) trans 4295069850 rc 0/0 00000100:00100000:0.0:1785220581.393873:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.393893:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.393907:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908736 00000100:00100000:0.0:1785220581.393911:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908736 00010000:00010000:0.0:1785220581.393929:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a1b9cab9/0xf62499ab92781bfd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7899f expref: 454 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.393949:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908992 00000100:00100000:0.0:1785220581.393952:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908992 00010000:00010000:0.0:1785220581.393965:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006448a006/0xf62499ab9278544a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a44f expref: 403 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.393983:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100013 00000100:00100000:0.0:1785220581.393990:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+454:2407:x1871938519908736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.393999:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.394004:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a1b9cab9/0xf62499ab92781bfd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7899f expref: 454 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394015:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a1b9cab9/0xf62499ab92781bfd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7899f expref: 454 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394029:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.394033:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.394037:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394046:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394059:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.394109:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+454:2407:x1871938519908736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.394117:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100013 00000100:00100000:0.0:1785220581.394131:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100014 00000100:00100000:0.0:1785220581.394138:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+403:2407:x1871938519908992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.394146:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.394151:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006448a006/0xf62499ab9278544a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a44f expref: 403 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394159:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006448a006/0xf62499ab9278544a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a44f expref: 403 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394169:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.394173:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.394176:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394183:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394197:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.394549:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+403:2407:x1871938519908992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (320us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.394558:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100014 00000100:00100000:0.0:1785220581.394608:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.394627:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519908864 00000100:00100000:0.0:1785220581.394632:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519908864 00010000:00010000:0.0:1785220581.394647:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007315256f/0xf62499ab92781b16 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7891a expref: 454 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.394669:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100015 00000100:00100000:0.0:1785220581.394676:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90702050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+454:2407:x1871938519908864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.394685:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.394690:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007315256f/0xf62499ab92781b16 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7891a expref: 454 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394701:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007315256f/0xf62499ab92781b16 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7891a expref: 454 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394713:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.394716:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.394720:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394728:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.394741:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.394760:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90702050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+454:2407:x1871938519908864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.394778:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100015 00000004:00080000:1.0:1785220581.394792:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6efb80/2, rc 0, transno 4295069850 00000100:00100000:1.0:1785220581.394803:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510599296:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.394810:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.394822:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510601600:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.394898:0:14175:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284077 00010000:00010000:2.0:1785220581.394929:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.394942:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.394951:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.394961:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.394977:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.395357:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.395390:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510601728:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.395398:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284078 00000100:00100000:1.0:1785220581.395414:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.395423:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510601600 00000100:00100000:3.0:1785220581.395428:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510601600 00000100:00100000:3.0:1785220581.395450:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510601728 00000100:00100000:3.0:1785220581.395453:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510601728 00000100:00100000:1.0:1785220581.395455:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510601344:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.395471:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284084 00000100:00100000:3.0:1785220581.395478:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1077:3653:x1871938510601600:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.395478:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:3.0:1785220581.395500:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29148 count 1 00000004:00080000:1.0:1785220581.395502:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6eca80/2, rc 0, transno 4295070450 00000100:00100000:1.0:1785220581.395510:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6eca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510600320:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.395520:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ed180/2, rc 0, transno 4295069852 00010000:00010000:3.0:1785220581.395521:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.395527:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510599040:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.395535:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c588006d/0xf62499ab9278559a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a4e9 expref: 402 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.395536:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97841880/2, rc 0, transno 4295070449 00000100:00100000:1.0:1785220581.395540:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510598400:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.395543:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71dc:0x0].0x0 since=30s v=(0 0) 00010000:00010000:3.0:1785220581.395546:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71dc:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.395548:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:3.0:1785220581.395550:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.395553:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510601472:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.395572:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.395574:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c588006d/0xf62499ab9278559a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a4e9 expref: 402 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.395586:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c588006d/0xf62499ab9278559a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a4e9 expref: 402 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.395595:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510601344 00000100:00100000:3.0:1785220581.395598:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510601856:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.395598:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510601344 00000100:00100000:1.0:1785220581.395618:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510601472 00000100:00100000:1.0:1785220581.395620:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510601472 00000100:00100000:1.0:1785220581.395635:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284085 00000100:00100000:3.0:1785220581.395638:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.395640:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1002:3653:x1871938510601728:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.395652:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29089 count 1 00010000:00010000:1.0:1785220581.395665:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.395668:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009dbd57b8/0xf62499ab92784f11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1af expref: 402 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.395676:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000034d56c00/0xf62499ab92781f53 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78b19 expref: 453 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.395680:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510599424:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.395682:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a1:0x0].0x0 since=33s v=(0 0) 00010000:00010000:1.0:1785220581.395685:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.395688:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.395697:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.395706:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.395706:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000034d56c00/0xf62499ab92781f53 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78b19 expref: 453 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.395715:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.395717:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000034d56c00/0xf62499ab92781f53 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78b19 expref: 453 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.395723:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.395727:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510601984:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.395731:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.395745:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.395748:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01c00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.395855:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8e00fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1078:3655:x1871938510597248:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14378us (15503us total) trans 4295070451 rc 0/0 00000100:00100000:2.0:1785220581.395934:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1002:3654:x1871938510598912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14001us (15340us total) trans 4295069853 rc 0/0 00000100:00100000:0.0:1785220581.396003:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510597888:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.396018:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.396071:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.396107:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519909120 00000100:00100000:0.0:1785220581.396110:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519909120 00010000:00010000:0.0:1785220581.396129:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007bfdfd21/0xf62499ab92784fe3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a218 expref: 402 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.396151:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100016 00000100:00100000:0.0:1785220581.396157:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+402:2407:x1871938519909120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.396167:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.396171:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007bfdfd21/0xf62499ab92784fe3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a218 expref: 402 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396182:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007bfdfd21/0xf62499ab92784fe3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a218 expref: 402 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396199:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007bfdfd21/0xf62499ab92784fe3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a218 expref: 402 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396210:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d5:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.396262:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284086 00000100:00100000:1.0:1785220581.396273:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1002:3654:x1871938510601344:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.396290:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29087 count 1 00010000:00010000:1.0:1785220581.396302:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.396314:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000065d45e7b/0xf62499ab92781cba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a0f expref: 453 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.396321:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x719f:0x0].0x0 since=33s v=(0 0) 00010000:00010000:1.0:1785220581.396323:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.396327:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.396346:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000065d45e7b/0xf62499ab92781cba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a0f expref: 453 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.396355:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000065d45e7b/0xf62499ab92781cba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a0f expref: 453 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.396365:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510602112:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.396396:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01900 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.396628:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.396635:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396644:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396666:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.396687:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+401:2407:x1871938519909120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 530us (616us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.396694:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100016 00010000:00010000:0.0:1785220581.396715:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396723:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396732:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396738:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.396746:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.396859:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1079:3655:x1871938510597376:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19030us (21750us total) trans 4295070452 rc 0/0 00000100:00100000:3.0:1785220581.397038:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284073 00000100:00100000:3.0:1785220581.397058:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284087 00000100:00100000:3.0:1785220581.397128:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1002:3654:x1871938510601472:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.397147:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29088 count 1 00010000:00010000:3.0:1785220581.397161:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397174:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000097602eca/0xf62499ab92781dfc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a86 expref: 453 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397181:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a0:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.397184:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.397188:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397210:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000097602eca/0xf62499ab92781dfc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a86 expref: 453 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397232:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000097602eca/0xf62499ab92781dfc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a86 expref: 453 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.397244:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510602240:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.397284:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.397333:0:14179:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a1b9cab9/0xf62499ab92781bfd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7899f expref: 453 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.397351:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510600832:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.397370:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397379:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397385:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397394:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397401:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397410:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220581.397514:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1003:3653:x1871938510600704:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12893us (13936us total) trans 4295069854 rc 0/0 00000100:00100000:3.0:1785220581.397521:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284081 00010000:00010000:3.0:1785220581.397550:0:14200:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007315256f/0xf62499ab92781b16 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7891a expref: 452 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.397560:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510600960:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.397572:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397579:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397589:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397597:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397603:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397610:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:3.0:1785220581.397667:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1003:3654:x1871938510600576:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12481us (14043us total) trans 4295069855 rc 0/0 00000100:00100000:3.0:1785220581.397673:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284082 00010000:00010000:3.0:1785220581.397698:0:14193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006448a006/0xf62499ab9278544a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a44f expref: 400 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.397712:0:14193:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510601216:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.397723:0:14193:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397735:0:14193:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397742:0:14193:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397749:0:14193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397755:0:14193:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397761:0:14193:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:3.0:1785220581.397834:0:14193:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1079:3653:x1871938510601088:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12117us (12249us total) trans 4295070453 rc 0/0 00000100:00100000:3.0:1785220581.397840:0:14193:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284083 00010000:00010000:3.0:1785220581.397861:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:3.0:1785220581.397867:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397875:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.397879:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397890:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397985:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.397991:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a46af lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7199:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398005:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:3.0:1785220581.398008:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398012:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.398014:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398019:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398024:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398028:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4685 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398038:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:3.0:1785220581.398040:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398045:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.398047:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398051:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398056:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398059:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4662 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398067:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.398069:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398072:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.398073:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398077:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398082:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398107:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a46bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398125:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.398129:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398138:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.398141:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398148:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398157:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398164:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a46c4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398177:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.398181:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398188:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.398191:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398262:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398274:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398280:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a46cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.398292:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.398294:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.398295:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.398376:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284079 00010000:00010000:1.0:1785220581.398398:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.398416:0:14166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb8ad840/0xf62499ab92785260 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a361 expref: 399 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.398422:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915afb80/2, rc 0, transno 4295070453 00000100:00100000:2.0:1785220581.398426:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510598272:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.398427:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510601088:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.398438:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a7800/2, rc 0, transno 4295069854 00010000:00010000:2.0:1785220581.398442:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.398442:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510600704:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.398450:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398456:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398463:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398469:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398491:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.398541:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.398552:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510602624:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.398576:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1079:3654:x1871938510597760:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19576us (21080us total) trans 4295070454 rc 0/0 00000100:00100000:1.0:1785220581.398580:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.398583:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284071 00000100:00100000:2.0:1785220581.398596:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510602624 00000100:00100000:2.0:1785220581.398602:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510602624 00000100:00100000:1.0:1785220581.398614:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510602368:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.398632:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284088 00000100:00100000:1.0:1785220581.398635:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.398639:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b033100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1079:3653:x1871938510602624:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.398655:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29151 count 1 00000004:00080000:1.0:1785220581.398655:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a5c00/2, rc 0, transno 4295069855 00000100:00100000:1.0:1785220581.398659:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510600576:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.398669:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.398669:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ec000/2, rc 0, transno 4295069853 00000100:00100000:1.0:1785220581.398672:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510598912:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.398679:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97843b80/2, rc 0, transno 4295070454 00010000:00010000:2.0:1785220581.398681:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b17b6832/0xf62499ab927857d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a60f expref: 398 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.398686:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97843b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510597760:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.398690:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71df:0x0].0x0 since=30s v=(0 0) 00000100:00100000:1.0:1785220581.398693:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:2.0:1785220581.398694:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.398698:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.398699:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510602496:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.398719:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.398720:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b17b6832/0xf62499ab927857d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a60f expref: 398 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398731:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b17b6832/0xf62499ab927857d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a60f expref: 398 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.398742:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510602752:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.398758:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa62d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510602880:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.398792:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.398792:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.398811:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398824:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.398826:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbf800/2, rc 0, transno 4295070451 00000100:00100000:1.0:1785220581.398832:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510597248:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.398833:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398840:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.398854:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1785220581.398921:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbdf80/2, rc 0, transno 4295070452 00000100:00100000:1.0:1785220581.398925:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510597376:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.398929:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1080:3653:x1871938510597632:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20252us (23532us total) trans 4295070455 rc 0/0 00000100:00100000:2.0:1785220581.398936:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284069 00000004:00080000:1.0:1785220581.398943:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9365d500/2, rc 0, transno 4295070455 00000100:00100000:1.0:1785220581.398946:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9365d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510597632:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.398948:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510602368 00000100:00100000:2.0:1785220581.398951:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510602368 00000100:00100000:2.0:1785220581.398971:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510602496 00000100:00100000:2.0:1785220581.398974:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510602496 00000100:00100000:2.0:1785220581.398989:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510602880 00000100:00100000:1.0:1785220581.398991:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.398992:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510602880 00000100:00100000:1.0:1785220581.398997:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510603264:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.399008:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284089 00000100:00100000:2.0:1785220581.399014:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1082:3654:x1871938510602368:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.399018:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:2.0:1785220581.399025:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29149 count 1 00010000:00010000:2.0:1785220581.399036:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399043:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa61880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603008:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.399048:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002f3d922d/0xf62499ab9278566c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a559 expref: 398 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399055:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71dd:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.399058:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71dd:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.399061:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.399062:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399077:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:2.0:1785220581.399081:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002f3d922d/0xf62499ab9278566c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a559 expref: 398 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399081:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603136:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.399108:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002f3d922d/0xf62499ab9278566c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a559 expref: 398 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399118:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.399119:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510603392:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.399147:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.399171:0:13837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b6b18d6a/0xf62499ab927815dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78665 expref: 451 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.399183:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510599552:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.399195:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399203:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399242:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9147ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510603520:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.399264:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.399317:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.399322:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399330:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffe00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.399334:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399338:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:3.0:1785220581.399342:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399345:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91479c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510603904:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.399355:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399363:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a4677 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399366:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.399380:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.399383:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399389:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91478000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603648:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.399390:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.399393:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399401:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399408:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.399411:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399419:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4669 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.399423:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.399427:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9147a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603776:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.399433:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.399435:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.399437:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.399445:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.399470:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399480:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399487:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399525:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.399608:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1002:3653:x1871938510598656:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18091us (19165us total) trans 4295069856 rc 0/0 00000100:00100000:2.0:1785220581.399615:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284074 00000100:00100000:2.0:1785220581.399629:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510603264 00000100:00100000:2.0:1785220581.399632:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510603264 00000004:00080000:1.0:1785220581.399633:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6edf80/2, rc 0, transno 4295069856 00000100:00100000:1.0:1785220581.399642:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510598656:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.399650:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510603008 00000100:00100000:2.0:1785220581.399653:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510603008 00000100:00100000:2.0:1785220581.399668:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510603136 00000100:00100000:2.0:1785220581.399671:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510603136 00000100:00100000:1.0:1785220581.399674:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510604032:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.399686:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510603520 00000100:00100000:2.0:1785220581.399689:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510603520 00000100:00100000:1.0:1785220581.399695:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.399740:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284068 00010000:00010000:3.0:1785220581.399775:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.399780:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399792:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.399795:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.399795:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510603904 00000100:00100000:2.0:1785220581.399799:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510603904 00010000:00010000:3.0:1785220581.399803:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399815:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.399820:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510603648 00000100:00100000:2.0:1785220581.399822:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510603648 00010000:00010000:3.0:1785220581.399823:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a468c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7197:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399839:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.399840:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.399842:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.399853:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510603776 00000100:00100000:3.0:1785220581.399857:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510603776 00000100:00100000:3.0:1785220581.399881:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510604032 00000100:00100000:3.0:1785220581.399884:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510604032 00000100:00100000:3.0:1785220581.399903:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284090 00000100:00100000:2.0:1785220581.399908:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284091 00000100:00100000:3.0:1785220581.399910:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1084:3654:x1871938510602496:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.399913:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1084:3653:x1871938510602880:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.399925:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29150 count 1 00002000:00080000:2.0:1785220581.399925:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29152 count 1 00010000:00010000:2.0:1785220581.399937:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399938:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399950:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004291f5e5/0xf62499ab927858e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a67f expref: 398 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399953:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008c16fc1d/0xf62499ab9278571b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5bb expref: 398 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399958:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e0:0x0].0x0 since=30s v=(0 0) 00010000:00010000:3.0:1785220581.399959:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71de:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.399961:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.399962:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.399966:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399966:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399986:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008c16fc1d/0xf62499ab9278571b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5bb expref: 398 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399986:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004291f5e5/0xf62499ab927858e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a67f expref: 398 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.399998:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008c16fc1d/0xf62499ab9278571b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5bb expref: 398 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.399998:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004291f5e5/0xf62499ab927858e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a67f expref: 398 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.400010:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510604288:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.400010:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:13837:1871938510604160:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.400031:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.400033:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.400050:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284092 00000100:00100000:2.0:1785220581.400057:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1084:3653:x1871938510603264:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.400069:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29155 count 1 00010000:00010000:2.0:1785220581.400079:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.400123:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002d2d6c2c/0xf62499ab92785c62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7f9 expref: 398 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.400129:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e3:0x0].0x0 since=30s v=(0 0) 00000100:00100000:3.0:1785220581.400151:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284093 00000100:00100000:3.0:1785220581.400160:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1084:3654:x1871938510603008:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.400173:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29153 count 1 00010000:00010000:3.0:1785220581.400183:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400195:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001d47c9eb/0xf62499ab92785a71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a720 expref: 398 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400209:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e1:0x0].0x0 since=30s v=(0 0) 00010000:00010000:3.0:1785220581.400211:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e1:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.400252:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284094 00000100:00100000:1.0:1785220581.400262:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9147b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1084:3654:x1871938510603136:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.400274:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29154 count 1 00010000:00010000:1.0:1785220581.400285:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400291:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400297:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002013cd90/0xf62499ab92785b74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a790 expref: 398 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400304:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e2:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.400306:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.400311:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001d47c9eb/0xf62499ab92785a71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a720 expref: 398 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400311:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400321:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001d47c9eb/0xf62499ab92785a71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a720 expref: 398 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400327:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002013cd90/0xf62499ab92785b74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a790 expref: 398 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.400331:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510604416:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.400336:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002013cd90/0xf62499ab92785b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a790 expref: 398 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.400345:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510604544:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.400356:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.400358:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.400372:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284095 00000100:00100000:3.0:1785220581.400380:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1006:3653:x1871938510603520:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.400387:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e3:0x0].0x0 since=0s v=(1 0) 00002000:00080000:3.0:1785220581.400392:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29090 count 1 00010000:00010000:2.0:1785220581.400392:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400405:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.400411:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002d2d6c2c/0xf62499ab92785c62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7f9 expref: 398 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400418:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c5803432/0xf62499ab927820c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ba5 expref: 450 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.400420:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002d2d6c2c/0xf62499ab92785c62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7f9 expref: 398 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400425:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a2:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.400428:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a2:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220581.400431:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510604672:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.400433:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.400448:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a5a9d80 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.400449:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c5803432/0xf62499ab927820c6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ba5 expref: 450 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400458:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c5803432/0xf62499ab927820c6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ba5 expref: 450 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.400468:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510604800:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.400479:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284096 00000100:00100000:1.0:1785220581.400487:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1006:3653:x1871938510603904:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.400493:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00002000:00080000:1.0:1785220581.400499:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29093 count 1 00010000:00010000:1.0:1785220581.400510:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400521:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fb530c4e/0xf62499ab92782366 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d42 expref: 450 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.400522:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284097 00010000:00010000:1.0:1785220581.400527:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a5:0x0].0x0 since=33s v=(0 0) 00000100:00100000:3.0:1785220581.400529:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1006:3654:x1871938510603648:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.400530:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.400534:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.400539:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29091 count 1 00010000:00010000:3.0:1785220581.400549:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400551:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fb530c4e/0xf62499ab92782366 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d42 expref: 450 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400560:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009e97965d/0xf62499ab927821b4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c23 expref: 450 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400561:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fb530c4e/0xf62499ab92782366 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d42 expref: 450 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400568:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a3:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.400571:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a3:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.400571:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510604928:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.400574:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400590:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009e97965d/0xf62499ab927821b4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c23 expref: 450 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.400591:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.400599:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009e97965d/0xf62499ab927821b4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c23 expref: 450 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.400606:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284098 00000100:00100000:3.0:1785220581.400608:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510605056:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.400613:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98adf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1006:3654:x1871938510603776:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.400624:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00002000:00080000:1.0:1785220581.400624:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29092 count 1 00010000:00010000:1.0:1785220581.400635:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.400638:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284099 00000100:00100000:3.0:1785220581.400647:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1006:3653:x1871938510604032:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.400647:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009bcd148b/0xf62499ab927822b7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78cc4 expref: 450 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400654:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a4:0x0].0x0 since=33s v=(0 0) 00002000:00080000:3.0:1785220581.400657:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29094 count 1 00010000:00010000:1.0:1785220581.400657:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.400661:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400666:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.400676:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009bcd148b/0xf62499ab927822b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78cc4 expref: 450 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400677:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000edfde40d/0xf62499ab9278244d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78dc0 expref: 450 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400684:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a6:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.400687:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.400687:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009bcd148b/0xf62499ab927822b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78cc4 expref: 450 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.400691:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.400697:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510605184:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.400707:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000edfde40d/0xf62499ab9278244d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78dc0 expref: 450 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.400713:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01f00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.400715:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000edfde40d/0xf62499ab9278244d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78dc0 expref: 450 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.400729:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510605312:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.400744:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.400791:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.400797:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.400803:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.403619:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510601856:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.403652:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.406239:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ece00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510602112:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.406271:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.406479:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510603392:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.406495:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.406515:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510601984:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.406530:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.406577:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510602240:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.406592:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.407679:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:13837:1871938510604160:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.407700:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.412834:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510604544:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.412834:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510604416:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.412867:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.412867:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.412949:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510604288:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.412963:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.412969:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510604928:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.412984:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.413031:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510604800:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.413044:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.413050:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510605184:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.413062:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.413124:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510605056:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.413148:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.413150:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.413167:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.413189:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519909248 00000100:00100000:1.0:1785220581.413196:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519909248 00000100:00100000:0.0:1785220581.413226:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510605312:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.413231:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c588006d/0xf62499ab9278559a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a4e9 expref: 399 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.413239:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.413256:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519909504 00000100:00100000:0.0:1785220581.413260:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519909504 00000100:00100000:1.0:1785220581.413263:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100017 00000100:00100000:1.0:1785220581.413271:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+399:2407:x1871938519909248:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.413277:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000097602eca/0xf62499ab92781dfc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a86 expref: 451 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.413284:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.413290:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c588006d/0xf62499ab9278559a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a4e9 expref: 399 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.413300:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c588006d/0xf62499ab9278559a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a4e9 expref: 399 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.413302:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100018 00000100:00100000:0.0:1785220581.413308:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+451:2407:x1871938519909504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.413316:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.413320:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000097602eca/0xf62499ab92781dfc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a86 expref: 451 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.413330:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c588006d/0xf62499ab9278559a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a4e9 expref: 399 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.413330:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000097602eca/0xf62499ab92781dfc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a86 expref: 451 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.413344:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000097602eca/0xf62499ab92781dfc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a86 expref: 451 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.413348:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.413352:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.413354:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.413356:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.413357:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.413361:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.413366:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.413368:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.413386:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.413397:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.413435:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+450:2407:x1871938519909504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 125us (266us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.413442:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100018 00010000:00010000:3.0:1785220581.413575:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.413584:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.413595:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.413603:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.413620:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.413686:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510604672:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.413701:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.413723:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510602752:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.413735:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.414233:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+398:2407:x1871938519909248:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 960us (1083us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.414242:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100017 00000100:00100000:0.0:1785220581.415073:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.415127:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.415279:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519909760 00000100:00100000:1.0:1785220581.415286:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519909760 00000100:00100000:0.0:1785220581.415312:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910016 00010000:00010000:1.0:1785220581.415313:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004291f5e5/0xf62499ab927858e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a67f expref: 398 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.415316:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910016 00010000:00010000:0.0:1785220581.415331:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008c16fc1d/0xf62499ab9278571b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5bb expref: 399 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.415340:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100019 00000100:00100000:1.0:1785220581.415354:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+399:2407:x1871938519909760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.415366:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.415371:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004291f5e5/0xf62499ab927858e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a67f expref: 399 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.415383:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004291f5e5/0xf62499ab927858e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a67f expref: 399 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.415404:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004291f5e5/0xf62499ab927858e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a67f expref: 399 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.415418:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.415422:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.415427:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.415436:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.415462:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.415494:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+398:2407:x1871938519909760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 146us (423us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.415511:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1007:3654:x1871938510601472:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18387us (19936us total) trans 4295069857 rc 0/0 00000100:00100000:3.0:1785220581.415520:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284087 00010000:00010000:3.0:1785220581.415542:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.415548:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415556:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.415559:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415573:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.415577:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.415584:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415586:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415593:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a46e7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.415594:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.415602:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415611:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.415614:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.415616:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220581.415620:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fea00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.415622:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415629:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415637:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415643:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a11d065a/0xf62499ab927a46fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415659:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.415661:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.415662:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.415673:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415680:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415688:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415695:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415706:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.415795:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1086:3653:x1871938510601600:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20317us (20437us total) trans 4295070457 rc 0/0 00000100:00100000:3.0:1785220581.415802:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284084 00010000:00010000:3.0:1785220581.415819:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.415824:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415831:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.415844:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415852:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415862:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415869:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a46d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.415882:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.415883:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.415885:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.415969:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1085:3653:x1871938510602880:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16055us (17175us total) trans 4295070456 rc 0/0 00000100:00100000:0.0:1785220581.415976:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100020 00000100:00100000:2.0:1785220581.415979:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284091 00000100:00100000:0.0:1785220581.415984:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+398:2407:x1871938519910016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.415992:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.415998:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008c16fc1d/0xf62499ab9278571b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5bb expref: 398 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416008:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008c16fc1d/0xf62499ab9278571b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5bb expref: 398 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416021:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c16fc1d/0xf62499ab9278571b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5bb expref: 398 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416030:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.416033:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.416037:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416044:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416147:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.416167:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2407:x1871938519910016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 185us (1039us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.416174:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100020 00000100:00100000:0.0:1785220581.416339:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.416356:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.416362:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910272 00000100:00100000:0.0:1785220581.416367:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910272 00010000:00010000:0.0:1785220581.416383:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002013cd90/0xf62499ab92785b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a790 expref: 397 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.416405:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100021 00000100:00100000:0.0:1785220581.416411:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2407:x1871938519910272:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.416419:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.416423:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002013cd90/0xf62499ab92785b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a790 expref: 397 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416433:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002013cd90/0xf62499ab92785b74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a790 expref: 397 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416446:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002013cd90/0xf62499ab92785b74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a790 expref: 397 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416455:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.416458:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.416462:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416471:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416487:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.416507:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+396:2407:x1871938519910272:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.416513:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100021 00000100:00100000:0.0:1785220581.416570:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.416596:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.416673:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.416688:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910528 00000100:00100000:0.0:1785220581.416692:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910528 00010000:00010000:0.0:1785220581.416706:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c5803432/0xf62499ab927820c6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ba5 expref: 450 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.416727:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910784 00000100:00100000:0.0:1785220581.416730:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910784 00010000:00010000:0.0:1785220581.416742:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009bcd148b/0xf62499ab927822b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78cc4 expref: 451 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.416778:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911040 00000100:00100000:0.0:1785220581.416781:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911040 00000100:00100000:2.0:1785220581.416784:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100022 00000100:00100000:2.0:1785220581.416791:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e82a2a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+452:2407:x1871938519910528:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.416794:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009e97965d/0xf62499ab927821b4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c23 expref: 452 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416800:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.416804:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c5803432/0xf62499ab927820c6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ba5 expref: 452 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.416812:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100023 00010000:00010000:2.0:1785220581.416815:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c5803432/0xf62499ab927820c6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ba5 expref: 452 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.416817:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+452:2407:x1871938519910784:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.416823:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.416828:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009bcd148b/0xf62499ab927822b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78cc4 expref: 452 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416830:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c5803432/0xf62499ab927820c6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ba5 expref: 452 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416840:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.416842:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.416844:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009bcd148b/0xf62499ab927822b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78cc4 expref: 451 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416847:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416855:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416856:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009bcd148b/0xf62499ab927822b7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78cc4 expref: 451 pid: 8442 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416866:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.416869:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.416872:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.416872:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416880:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.416893:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e82a2a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+450:2407:x1871938519910528:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (323us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.416894:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.416903:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100022 00000100:00100000:0.0:1785220581.416908:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+450:2407:x1871938519910784:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.416914:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100023 00000100:00100000:2.0:1785220581.416918:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100024 00000100:00100000:2.0:1785220581.416924:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+448:2407:x1871938519911040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.416932:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.416933:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416936:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009e97965d/0xf62499ab927821b4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c23 expref: 448 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416942:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416946:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009e97965d/0xf62499ab927821b4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c23 expref: 448 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416957:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416959:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e97965d/0xf62499ab927821b4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c23 expref: 448 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416965:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416969:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.416972:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.416976:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.416977:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.416983:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.416998:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.417015:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+447:2407:x1871938519911040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.417021:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100024 00010000:00010000:2.0:1785220581.417039:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.417047:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.417054:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.417061:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.417073:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.417073:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1007:3653:x1871938510603520:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16694us (17807us total) trans 4295069858 rc 0/0 00000100:00100000:0.0:1785220581.417080:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284095 00010000:00010000:0.0:1785220581.417129:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417138:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417146:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417153:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.417160:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1007:3654:x1871938510603648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16632us (17751us total) trans 4295069859 rc 0/0 00010000:00010000:0.0:1785220581.417165:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.417167:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284097 00000100:00100000:0.0:1785220581.417498:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98adf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1007:3654:x1871938510603776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16885us (18051us total) trans 4295069860 rc 0/0 00000100:00100000:0.0:1785220581.417508:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284098 00000100:00100000:0.0:1785220581.417547:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.417583:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.417617:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.417643:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.417658:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519909376 00000100:00100000:0.0:1785220581.417661:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519909376 00010000:00010000:0.0:1785220581.417677:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000034d56c00/0xf62499ab92781f53 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78b19 expref: 447 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.417696:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519909632 00000100:00100000:0.0:1785220581.417699:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519909632 00010000:00010000:0.0:1785220581.417712:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002f3d922d/0xf62499ab9278566c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a559 expref: 396 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.417728:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519909888 00000100:00100000:0.0:1785220581.417731:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519909888 00010000:00010000:0.0:1785220581.417742:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000065d45e7b/0xf62499ab92781cba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a0f expref: 448 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.417757:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910144 00000100:00100000:0.0:1785220581.417760:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910144 00010000:00010000:0.0:1785220581.417791:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001d47c9eb/0xf62499ab92785a71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a720 expref: 397 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.417812:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100025 00000100:00100000:0.0:1785220581.417818:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+448:2407:x1871938519909376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.417826:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.417830:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000034d56c00/0xf62499ab92781f53 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78b19 expref: 448 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417900:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000034d56c00/0xf62499ab92781f53 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78b19 expref: 448 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417915:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034d56c00/0xf62499ab92781f53 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78b19 expref: 448 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417925:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.417928:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.417932:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417939:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.417957:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.417973:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100019 00000100:00100000:0.0:1785220581.417976:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+447:2407:x1871938519909376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 159us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.417982:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100025 00000100:00100000:0.0:1785220581.417993:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100026 00000100:00100000:0.0:1785220581.417998:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+396:2407:x1871938519909632:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.418005:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.418010:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002f3d922d/0xf62499ab9278566c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a559 expref: 396 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.418013:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100027 00000100:00100000:1.0:1785220581.418020:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+446:2407:x1871938519909888:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.418021:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002f3d922d/0xf62499ab9278566c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a559 expref: 396 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418027:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.418031:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000065d45e7b/0xf62499ab92781cba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a0f expref: 446 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.418033:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f3d922d/0xf62499ab9278566c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a559 expref: 396 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.418044:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.418046:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000065d45e7b/0xf62499ab92781cba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a0f expref: 446 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.418047:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.418052:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418058:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000065d45e7b/0xf62499ab92781cba lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78a0f expref: 446 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.418059:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418068:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.418071:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x719f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.418075:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.418075:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.418083:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418221:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.418288:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418298:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418306:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418313:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418334:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:1.0:1785220581.418563:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+445:2407:x1871938519909888:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 544us (945us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.418563:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+395:2407:x1871938519909632:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 562us (978us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.418571:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100026 00000100:00100000:1.0:1785220581.418574:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100027 00000100:00100000:1.0:1785220581.418585:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100028 00000100:00100000:1.0:1785220581.418591:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+394:2407:x1871938519910144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.418598:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.418603:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001d47c9eb/0xf62499ab92785a71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a720 expref: 394 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418617:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001d47c9eb/0xf62499ab92785a71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a720 expref: 394 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418629:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d47c9eb/0xf62499ab92785a71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a720 expref: 394 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418639:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.418642:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.418646:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418652:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.418670:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.418690:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+393:2407:x1871938519910144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (1045us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.418739:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1085:3654:x1871938510602368:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19724us (20102us total) trans 4295070458 rc 0/0 00000100:00100000:2.0:1785220581.418747:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284089 00010000:00010000:2.0:1785220581.418776:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220581.418782:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418790:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.418794:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418803:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418815:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418823:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a471f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418842:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.418846:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418857:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.418860:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418867:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418876:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418883:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a472d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418898:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.418902:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418908:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.418911:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418918:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418928:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418937:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4734 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418950:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.418953:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418959:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.418962:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418970:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418980:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.418988:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a46f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.419001:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.419003:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.419005:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.419823:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.419894:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.419909:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.419918:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.419941:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.419945:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.419949:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.419957:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.419960:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.419967:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.419980:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.419986:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a46d9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420000:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.420001:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.420003:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.420064:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1007:3653:x1871938510601728:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24422us (24648us total) trans 4295069861 rc 0/0 00000100:00100000:0.0:1785220581.420072:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284085 00010000:00010000:3.0:1785220581.420366:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420376:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420385:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420392:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420412:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.420426:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.420430:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420438:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.420441:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420448:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420460:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420466:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4703 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420481:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.420483:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.420485:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.420493:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1085:3654:x1871938510602496:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20584us (21772us total) trans 4295070459 rc 0/0 00000100:00100000:3.0:1785220581.420501:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284090 00010000:00010000:3.0:1785220581.420529:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420539:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420547:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420555:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.420572:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.420576:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.420580:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420587:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.420591:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420598:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420609:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420616:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a46e0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x719f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.420633:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.420634:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.420636:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.420641:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1007:3654:x1871938510601344:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24371us (25162us total) trans 4295069862 rc 0/0 00000100:00100000:3.0:1785220581.420648:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284086 00010000:00010000:3.0:1785220581.420665:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.420672:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.420676:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.420744:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.420754:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.420774:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.420783:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.420805:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.420954:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1085:3654:x1871938510603008:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20795us (21891us total) trans 4295070460 rc 0/0 00000100:00100000:0.0:1785220581.420964:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284093 00000100:00100000:0.0:1785220581.421032:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.421055:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910400 00000100:00100000:0.0:1785220581.421062:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910400 00010000:00010000:3.0:1785220581.421337:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.421342:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421350:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.421353:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421362:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421375:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421383:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4711 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421399:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.421401:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.421403:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.421436:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002d2d6c2c/0xf62499ab92785c62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7f9 expref: 394 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.421467:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100029 00000100:00100000:0.0:1785220581.421475:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+394:2407:x1871938519910400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.421490:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.421496:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002d2d6c2c/0xf62499ab92785c62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7f9 expref: 394 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.421506:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002d2d6c2c/0xf62499ab92785c62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7f9 expref: 394 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.421524:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d2d6c2c/0xf62499ab92785c62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7f9 expref: 394 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.421536:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.421540:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.421546:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.421557:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.421624:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.421655:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+393:2407:x1871938519910400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 181us (624us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.421663:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100029 00000100:00100000:0.0:1785220581.421727:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.421749:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910656 00000100:00100000:2.0:1785220581.421754:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910656 00000100:00100000:0.0:1785220581.421777:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.421794:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519910912 00000100:00100000:0.0:1785220581.421799:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519910912 00010000:00010000:0.0:1785220581.421815:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000edfde40d/0xf62499ab9278244d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78dc0 expref: 446 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421908:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421917:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421925:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421932:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.421949:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.422042:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1085:3653:x1871938510603264:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21985us (23021us total) trans 4295070461 rc 0/0 00000100:00100000:3.0:1785220581.422049:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284092 00010000:00010000:3.0:1785220581.422066:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.422070:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.422077:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.422080:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.422291:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.422295:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fb530c4e/0xf62499ab92782366 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d42 expref: 446 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.422303:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.422323:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100030 00000100:00100000:1.0:1785220581.422336:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100028 00000100:00100000:0.0:1785220581.422336:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+446:2407:x1871938519910912:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.422347:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.422351:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000edfde40d/0xf62499ab9278244d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78dc0 expref: 446 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422360:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000edfde40d/0xf62499ab9278244d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78dc0 expref: 446 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422372:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edfde40d/0xf62499ab9278244d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78dc0 expref: 446 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422382:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.422385:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422386:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.422391:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.422393:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422399:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.422402:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.422410:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422417:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.422423:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.422437:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+445:2407:x1871938519910912:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (660us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.422443:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100030 00010000:00010000:0.0:1785220581.422461:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422470:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422477:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422484:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.422496:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.422498:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1085:3654:x1871938510603136:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22239us (23378us total) trans 4295070462 rc 0/0 00000100:00100000:1.0:1785220581.422505:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284094 00000004:00080000:1.0:1785220581.422556:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9147ad80/2, rc 0, transno 4295069858 00000100:00100000:1.0:1785220581.422568:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9147ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510603520:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.422571:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1007:3653:x1871938510604032:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21927us (22876us total) trans 4295069863 rc 0/0 00000100:00100000:0.0:1785220581.422577:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284099 00000004:00080000:1.0:1785220581.422580:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c681880/2, rc 0, transno 4295070461 00000100:00100000:1.0:1785220581.422587:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510603264:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.422597:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa62d80/2, rc 0, transno 4295070456 00000100:00100000:1.0:1785220581.422600:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa62d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510602880:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.422609:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93665880/2, rc 0, transno 4295069861 00000100:00100000:1.0:1785220581.422613:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510601728:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.422622:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ad500/2, rc 0, transno 4295070457 00000100:00100000:1.0:1785220581.422625:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510601600:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.422635:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72dc00/2, rc 0, transno 4295069863 00000100:00100000:1.0:1785220581.422638:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510604032:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.422643:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.422705:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.422716:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510605696:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.422746:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.422793:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510605440:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.422817:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.422839:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9147a300/2, rc 0, transno 4295069860 00000100:00100000:1.0:1785220581.422844:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9147a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603776:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.422852:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91478000/2, rc 0, transno 4295069859 00000100:00100000:1.0:1785220581.422855:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91478000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603648:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.422863:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa60e00/2, rc 0, transno 4295070462 00000100:00100000:1.0:1785220581.422869:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603136:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.422878:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa61880/2, rc 0, transno 4295070460 00000100:00100000:1.0:1785220581.422881:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa61880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510603008:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.422889:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b031f80/2, rc 0, transno 4295070459 00000100:00100000:1.0:1785220581.422893:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510602496:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.422901:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97842a00/2, rc 0, transno 4295070458 00000100:00100000:1.0:1785220581.422904:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510602368:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.422912:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93666d80/2, rc 0, transno 4295069857 00000100:00100000:1.0:1785220581.422916:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510601472:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.422923:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93667b80/2, rc 0, transno 4295069862 00000100:00100000:1.0:1785220581.422927:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510601344:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.422936:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.422940:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510605568:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.422962:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.423028:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510605824:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.423049:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.423067:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.423071:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510606208:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.423185:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911168 00000100:00100000:2.0:1785220581.423189:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911168 00010000:00010000:2.0:1785220581.423207:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b17b6832/0xf62499ab927857d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a60f expref: 392 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.423230:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100031 00000100:00100000:2.0:1785220581.423237:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+444:2407:x1871938519910656:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.423246:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.423250:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fb530c4e/0xf62499ab92782366 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d42 expref: 444 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423260:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fb530c4e/0xf62499ab92782366 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d42 expref: 444 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423275:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fb530c4e/0xf62499ab92782366 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78d42 expref: 444 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423291:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.423295:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.423299:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423308:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423327:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.423350:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+443:2407:x1871938519910656:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (1624us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.423356:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100031 00000100:00100000:2.0:1785220581.423370:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100032 00000100:00100000:2.0:1785220581.423376:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+392:2407:x1871938519911168:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.423384:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.423388:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b17b6832/0xf62499ab927857d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a60f expref: 392 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423396:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b17b6832/0xf62499ab927857d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a60f expref: 392 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423410:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b17b6832/0xf62499ab927857d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a60f expref: 392 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423426:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.423429:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.423433:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423440:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423456:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.423474:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+391:2407:x1871938519911168:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (831us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.423480:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100032 00000100:00100000:2.0:1785220581.423509:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510605696 00000100:00100000:2.0:1785220581.423513:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510605696 00000100:00100000:2.0:1785220581.423534:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510605440 00000100:00100000:2.0:1785220581.423537:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510605440 00000100:00100000:2.0:1785220581.423553:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510605568 00000100:00100000:2.0:1785220581.423555:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510605568 00000100:00100000:2.0:1785220581.423569:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510605824 00000100:00100000:2.0:1785220581.423572:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510605824 00000100:00100000:2.0:1785220581.423588:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284100 00000100:00100000:2.0:1785220581.423594:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1088:3653:x1871938510605696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.423617:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29158 count 1 00010000:00010000:2.0:1785220581.423637:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423651:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000028005124/0xf62499ab92785f5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a93b expref: 390 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423659:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e6:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.423662:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.423666:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423689:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000028005124/0xf62499ab92785f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a93b expref: 390 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.423702:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000028005124/0xf62499ab92785f5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a93b expref: 390 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.423713:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510606336:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.423751:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.423989:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.424045:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510606208 00000100:00100000:1.0:1785220581.424049:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510606208 00000100:00100000:1.0:1785220581.424081:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284101 00000100:00100000:0.0:1785220581.424140:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284102 00000100:00100000:1.0:1785220581.424147:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d935a2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1089:3654:x1871938510605440:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.424149:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1089:3654:x1871938510605568:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.424161:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29156 count 1 00002000:00080000:0.0:1785220581.424161:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29157 count 1 00010000:00010000:1.0:1785220581.424174:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.424174:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.424187:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009709dc24/0xf62499ab92785da4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a85b expref: 390 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.424187:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e62fcd50/0xf62499ab92785ea0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e0 expref: 390 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.424195:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e4:0x0].0x0 since=30s v=(0 0) 00010000:00010000:0.0:1785220581.424195:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e5:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.424199:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.424199:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.424203:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.424204:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.424222:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009709dc24/0xf62499ab92785da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a85b expref: 390 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.424232:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009709dc24/0xf62499ab92785da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a85b expref: 390 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.424241:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510606464:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.424271:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01d80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.424304:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510605952:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.424340:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e62fcd50/0xf62499ab92785ea0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e0 expref: 390 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.424348:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.424350:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e62fcd50/0xf62499ab92785ea0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e0 expref: 390 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.424360:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510606592:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.424369:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.424374:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510606080:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.424381:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd590c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.424393:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.424524:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3137545/0xf62499ab927a470a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.424590:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.424615:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426510:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.426527:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426547:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426569:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426579:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4718 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426608:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.426613:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426623:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.426626:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426637:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426647:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.426658:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a473b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.427034:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.427037:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.427039:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.427371:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510605952 00000100:00100000:1.0:1785220581.427380:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510606080 00000100:00100000:0.0:1785220581.427380:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510605952 00000100:00100000:1.0:1785220581.427384:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510606080 00000100:00100000:1.0:1785220581.427441:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284103 00000100:00100000:0.0:1785220581.427446:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284104 00000100:00100000:1.0:1785220581.427450:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1091:3653:x1871938510605824:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.427452:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1091:3653:x1871938510606208:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.427478:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29162 count 1 00002000:00080000:1.0:1785220581.427491:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29159 count 1 00010000:00010000:0.0:1785220581.427498:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427501:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427517:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003af84692/0xf62499ab92786059 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9ab expref: 390 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427517:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f0941834/0xf62499ab92786297 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aae6 expref: 390 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427526:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e7:0x0].0x0 since=30s v=(0 0) 00010000:00010000:0.0:1785220581.427527:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ea:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.427531:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.427531:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.427536:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427536:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427567:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003af84692/0xf62499ab92786059 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9ab expref: 390 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427567:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f0941834/0xf62499ab92786297 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aae6 expref: 390 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427582:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003af84692/0xf62499ab92786059 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9ab expref: 390 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427582:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f0941834/0xf62499ab92786297 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aae6 expref: 390 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.427597:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510606848:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.427597:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510606720:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.427676:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df019c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.427686:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.427697:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284105 00000100:00100000:0.0:1785220581.427701:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284106 00000100:00100000:1.0:1785220581.427704:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1091:3654:x1871938510605952:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.427707:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1091:3654:x1871938510606080:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.427717:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29160 count 1 00002000:00080000:0.0:1785220581.427717:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29161 count 1 00010000:00010000:1.0:1785220581.427728:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427728:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427741:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e35c92ef/0xf62499ab9278614e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa0d expref: 390 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427745:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fc7321ab/0xf62499ab927861da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa68 expref: 390 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427748:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e8:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.427751:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.427752:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71e9:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.427755:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427755:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.427758:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427786:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e35c92ef/0xf62499ab9278614e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa0d expref: 390 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.427796:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e35c92ef/0xf62499ab9278614e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa0d expref: 390 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.427807:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510606976:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.427816:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fc7321ab/0xf62499ab927861da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa68 expref: 390 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.427825:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fc7321ab/0xf62499ab927861da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa68 expref: 390 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.427830:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.427901:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510607232:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.427944:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.428020:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.428027:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510607616:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.428049:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.428071:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510607232 00000100:00100000:1.0:1785220581.428075:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510607232 00000100:00100000:0.0:1785220581.428142:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510607104:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.428168:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd596c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.428182:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510607616 00000100:00100000:0.0:1785220581.428186:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510607616 00000100:00100000:2.0:1785220581.428205:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284107 00000100:00100000:2.0:1785220581.428214:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1008:3653:x1871938510607616:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.428227:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29098 count 1 00010000:00010000:2.0:1785220581.428242:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428255:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000073b09e1c/0xf62499ab92782787 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f64 expref: 442 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428261:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71aa:0x0].0x0 since=33s v=(0 0) 00010000:00010000:2.0:1785220581.428264:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.428269:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428289:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000073b09e1c/0xf62499ab92782787 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f64 expref: 442 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428298:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000073b09e1c/0xf62499ab92782787 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f64 expref: 442 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.428309:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510607744:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.428338:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.428360:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428368:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428376:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428384:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.428400:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.428469:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.428488:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911296 00000100:00100000:0.0:1785220581.428492:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911296 00010000:00010000:0.0:1785220581.428510:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000028005124/0xf62499ab92785f5d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a93b expref: 391 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.428532:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100033 00000100:00100000:0.0:1785220581.428539:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+391:2407:x1871938519911296:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.428548:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.428553:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000028005124/0xf62499ab92785f5d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a93b expref: 391 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.428564:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000028005124/0xf62499ab92785f5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a93b expref: 391 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.428580:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284108 00010000:00010000:0.0:1785220581.428582:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.428585:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e6:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.428587:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1009:3653:x1871938510607232:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.428590:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220581.428598:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29095 count 1 00010000:00010000:0.0:1785220581.428599:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.428609:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.428612:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.428621:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000c903c40/0xf62499ab927824ee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e3e expref: 442 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.428627:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a7:0x0].0x0 since=33s v=(0 0) 00010000:00010000:1.0:1785220581.428630:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a7:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.428630:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+391:2407:x1871938519911296:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (162us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.428634:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.428637:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100033 00010000:00010000:1.0:1785220581.428651:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000c903c40/0xf62499ab927824ee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e3e expref: 442 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.428661:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000c903c40/0xf62499ab927824ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e3e expref: 442 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.428671:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510607872:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.428694:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.428762:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91479c00/2, rc 0, transno 4295069864 00000100:00100000:1.0:1785220581.428782:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91479c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510603904:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.428803:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510607360:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.429591:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.429602:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.429614:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.429620:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.429630:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.429647:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.429658:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4726 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.429681:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.429683:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.429685:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.429691:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.429700:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.429792:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1009:3653:x1871938510603904:12345-0@lo:6:osp-syn-1-0.0 Request processed in 29302us (30421us total) trans 4295069864 rc 0/0 00000100:00100000:2.0:1785220581.429805:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284096 00010000:00010000:2.0:1785220581.429835:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.429844:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.429853:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.429860:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.429887:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.430021:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b033100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1092:3653:x1871938510602624:12345-0@lo:6:osp-syn-0-0.0 Request processed in 31381us (31440us total) trans 4295070463 rc 0/0 00000100:00100000:2.0:1785220581.430027:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284088 00010000:00010000:2.0:1785220581.430630:0:14163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000028005124/0xf62499ab92785f5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a93b expref: 390 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.430645:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510606336:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.430665:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.430675:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.430682:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.430690:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.430697:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.430712:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.431566:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1092:3653:x1871938510605696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7969us (8816us total) trans 4295070464 rc 0/0 00000100:00100000:2.0:1785220581.431576:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284100 00010000:00010000:3.0:1785220581.433922:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.433936:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.433950:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fee00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.433956:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.433967:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.433988:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.433996:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a46ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.434014:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.434018:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.434024:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.434027:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.434040:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.434049:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.434055:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4742 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.434067:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.434069:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.434071:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.434162:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.434341:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510608384:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.434387:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.434422:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.434428:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510608000:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.434448:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.434508:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510606464:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.434532:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.434571:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510608384 00000100:00100000:1.0:1785220581.434577:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510608384 00000100:00100000:1.0:1785220581.434616:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510608000 00000100:00100000:1.0:1785220581.434619:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510608000 00000100:00100000:1.0:1785220581.434640:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284109 00000100:00100000:1.0:1785220581.434649:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1010:3652:x1871938510608384:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.434674:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29102 count 1 00010000:00010000:1.0:1785220581.434697:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.434719:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f86d3395/0xf62499ab92782b31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7910f expref: 442 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.434728:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ae:0x0].0x0 since=33s v=(0 0) 00010000:00010000:1.0:1785220581.434733:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.434738:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.434778:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f86d3395/0xf62499ab92782b31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7910f expref: 442 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.434792:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f86d3395/0xf62499ab92782b31 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7910f expref: 442 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.434806:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510608512:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.434954:0:14169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284110 00000100:00100000:3.0:1785220581.434965:0:14169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1010:3652:x1871938510608000:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.434979:0:14169:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29099 count 1 00010000:00010000:3.0:1785220581.434994:0:14169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.435007:0:14169:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004cee371a/0xf62499ab9278287c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78fb1 expref: 442 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.435015:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ab:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.435018:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.435022:0:14169:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.435043:0:14169:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004cee371a/0xf62499ab9278287c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78fb1 expref: 442 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.435052:0:14169:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004cee371a/0xf62499ab9278287c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78fb1 expref: 442 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.435063:0:14169:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510608640:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.436176:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01d80 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.436251:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72c000/2, rc 0, transno 4295070464 00000100:00100000:1.0:1785220581.436264:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510605696:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.436276:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b030700/2, rc 0, transno 4295070463 00000100:00100000:1.0:1785220581.436280:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510602624:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.436348:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510608768:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.436392:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.436439:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510607488:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.436485:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.436512:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510608256:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.436532:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.436549:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1785220581.436554:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510608128:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.436582:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.436600:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510608896:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.436618:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.436646:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.436791:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510608768 00000100:00100000:2.0:1785220581.436797:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510608768 00000100:00100000:2.0:1785220581.437081:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510607488 00000100:00100000:2.0:1785220581.437287:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510607488 00000100:00100000:2.0:1785220581.437358:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510608256 00000100:00100000:2.0:1785220581.437362:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510608256 00000100:00100000:2.0:1785220581.437377:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510608128 00000100:00100000:2.0:1785220581.437379:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510608128 00000100:00100000:2.0:1785220581.437393:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510608896 00000100:00100000:2.0:1785220581.437395:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510608896 00000100:00100000:2.0:1785220581.437408:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510607360 00000100:00100000:2.0:1785220581.437410:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510607360 00000100:00100000:2.0:1785220581.437429:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284111 00000100:00100000:2.0:1785220581.437437:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1093:3653:x1871938510608768:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.437464:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29163 count 1 00010000:00010000:2.0:1785220581.437485:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.437506:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000227071a4/0xf62499ab9278641f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab80 expref: 389 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.437515:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71eb:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.437519:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.437524:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.437553:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000227071a4/0xf62499ab9278641f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab80 expref: 389 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.437566:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000227071a4/0xf62499ab9278641f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab80 expref: 389 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.437579:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510609024:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.437806:0:14169:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920c00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.437826:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284112 00000100:00100000:3.0:1785220581.437835:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1014:3655:x1871938510607488:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.437848:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29097 count 1 00010000:00010000:3.0:1785220581.437863:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.437875:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000988e10ec/0xf62499ab92782668 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ee6 expref: 442 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.437883:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a9:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.437886:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.437890:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.437916:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000988e10ec/0xf62499ab92782668 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ee6 expref: 442 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.437931:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000988e10ec/0xf62499ab92782668 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ee6 expref: 442 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.437942:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510609152:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.437965:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.437985:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284113 00000100:00100000:3.0:1785220581.437992:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1014:3655:x1871938510608256:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.438003:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29101 count 1 00010000:00010000:3.0:1785220581.438015:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438027:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007f0e66be/0xf62499ab92782a3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7907c expref: 442 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438033:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ad:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.438036:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.438039:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438055:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007f0e66be/0xf62499ab92782a3c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7907c expref: 442 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438063:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007f0e66be/0xf62499ab92782a3c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7907c expref: 442 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.438078:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510609280:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.438114:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.438131:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284114 00000100:00100000:3.0:1785220581.438139:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1014:3655:x1871938510608128:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.438151:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29100 count 1 00010000:00010000:3.0:1785220581.438162:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438173:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006b91fab2/0xf62499ab92782939 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7901a expref: 442 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438180:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ac:0x0].0x0 since=33s v=(0 0) 00010000:00010000:3.0:1785220581.438182:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.438186:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438205:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006b91fab2/0xf62499ab92782939 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7901a expref: 442 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438215:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006b91fab2/0xf62499ab92782939 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7901a expref: 442 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.438224:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510609408:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.438248:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a5a9d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.438264:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284115 00000100:00100000:3.0:1785220581.438272:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1093:3655:x1871938510608896:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.438288:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29164 count 1 00010000:00010000:3.0:1785220581.438299:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438310:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ec27ae0/0xf62499ab9278648f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abd4 expref: 389 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438317:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ec:0x0].0x0 since=30s v=(0 0) 00010000:00010000:3.0:1785220581.438322:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.438326:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438342:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ec27ae0/0xf62499ab9278648f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abd4 expref: 389 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.438350:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ec27ae0/0xf62499ab9278648f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abd4 expref: 389 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.438360:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510609536:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.438377:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.438554:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.438572:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284116 00000100:00100000:2.0:1785220581.438579:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1014:3654:x1871938510607360:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.438591:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29096 count 1 00010000:00010000:2.0:1785220581.438602:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.438618:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000054875f6c/0xf62499ab92782557 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e84 expref: 442 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.438624:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71a8:0x0].0x0 since=33s v=(0 0) 00010000:00010000:2.0:1785220581.438627:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.438630:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.438648:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000054875f6c/0xf62499ab92782557 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e84 expref: 442 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.438657:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000054875f6c/0xf62499ab92782557 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e84 expref: 442 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.438667:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510609664:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.438690:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510607104:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.438711:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.438788:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e85436c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.438874:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.438894:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.438911:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911552 00000100:00100000:0.0:1785220581.438915:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911552 00010000:00010000:0.0:1785220581.438951:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003af84692/0xf62499ab92786059 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9ab expref: 390 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.438974:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911808 00000100:00100000:0.0:1785220581.438978:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911808 00010000:00010000:0.0:1785220581.438988:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fc7321ab/0xf62499ab927861da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa68 expref: 391 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.439007:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100034 00000100:00100000:0.0:1785220581.439014:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+391:2407:x1871938519911552:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.439022:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.439028:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003af84692/0xf62499ab92786059 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9ab expref: 391 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439040:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003af84692/0xf62499ab92786059 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9ab expref: 391 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439060:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.439063:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.439068:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439077:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439379:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.439407:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+391:2407:x1871938519911552:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 393us (533us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.439415:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100034 00000100:00100000:0.0:1785220581.439435:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100035 00000100:00100000:0.0:1785220581.439441:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+390:2407:x1871938519911808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.439466:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.439472:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fc7321ab/0xf62499ab927861da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa68 expref: 390 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439483:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fc7321ab/0xf62499ab927861da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa68 expref: 390 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439495:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fc7321ab/0xf62499ab927861da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa68 expref: 390 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439506:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.439509:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.439513:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439521:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439542:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.439561:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+389:2407:x1871938519911808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 120us (665us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.439568:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100035 00010000:00010000:0.0:1785220581.439591:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439600:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439609:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439618:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439633:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.439778:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1094:3654:x1871938510606080:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12069us (15383us total) trans 4295070465 rc 0/0 00000100:00100000:0.0:1785220581.439786:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284106 00000100:00100000:0.0:1785220581.439830:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.439846:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912064 00000100:00100000:0.0:1785220581.439850:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912064 00010000:00010000:0.0:1785220581.439867:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000c903c40/0xf62499ab927824ee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e3e expref: 443 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.439895:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100036 00000100:00100000:0.0:1785220581.439901:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90701450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+443:2407:x1871938519912064:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.439908:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.439913:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000c903c40/0xf62499ab927824ee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e3e expref: 443 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439937:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000c903c40/0xf62499ab927824ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e3e expref: 443 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439951:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.439954:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.439958:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439967:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.439978:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.439992:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90701450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+443:2407:x1871938519912064:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.439999:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100036 00000100:00100000:0.0:1785220581.440048:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.440070:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.440111:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911424 00000100:00100000:0.0:1785220581.440115:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911424 00010000:00010000:0.0:1785220581.440143:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009709dc24/0xf62499ab92785da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a85b expref: 389 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.440878:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911680 00000100:00100000:0.0:1785220581.440885:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911680 00000100:00100000:2.0:1785220581.440886:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100037 00000100:00100000:2.0:1785220581.440894:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+390:2407:x1871938519911424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.440902:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e35c92ef/0xf62499ab9278614e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa0d expref: 390 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.440904:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.440909:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009709dc24/0xf62499ab92785da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a85b expref: 390 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.440919:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009709dc24/0xf62499ab92785da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a85b expref: 390 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.440924:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100038 00000100:00100000:0.0:1785220581.440930:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+390:2407:x1871938519911680:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.440935:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009709dc24/0xf62499ab92785da4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a85b expref: 390 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.440937:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.440941:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e35c92ef/0xf62499ab9278614e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa0d expref: 390 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.440946:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.440949:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.440950:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e35c92ef/0xf62499ab9278614e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa0d expref: 389 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.440954:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.440962:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.440963:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.440965:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.440969:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.440976:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.440984:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.440985:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.441071:0:14227:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003af84692/0xf62499ab92786059 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9ab expref: 389 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.441096:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510606848:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.441288:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+388:2407:x1871938519911680:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 357us (1217us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.441297:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100038 00000100:00100000:0.0:1785220581.441344:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.441387:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.441392:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510606592:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.441398:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.441404:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441405:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441415:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441428:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441444:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.441568:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1094:3653:x1871938510605824:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14119us (18517us total) trans 4295070466 rc 0/0 00000100:00100000:1.0:1785220581.441576:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284103 00010000:00010000:1.0:1785220581.441612:0:14267:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e35c92ef/0xf62499ab9278614e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa0d expref: 387 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.441623:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510606976:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.441636:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441644:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441651:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441659:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441666:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441681:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220581.441755:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1094:3654:x1871938510605952:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14051us (17406us total) trans 4295070467 rc 0/0 00000100:00100000:1.0:1785220581.441773:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284105 00000100:00100000:1.0:1785220581.441813:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519911936 00000100:00100000:1.0:1785220581.441818:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519911936 00010000:00010000:1.0:1785220581.441838:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000073b09e1c/0xf62499ab92782787 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f64 expref: 443 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.441865:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100039 00000100:00100000:1.0:1785220581.441873:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+443:2407:x1871938519911936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.441884:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.441887:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000073b09e1c/0xf62499ab92782787 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f64 expref: 443 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441897:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000073b09e1c/0xf62499ab92782787 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f64 expref: 443 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441917:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.441921:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.441926:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441935:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.441950:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.441965:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+443:2407:x1871938519911936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (621us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.441972:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100039 00000100:00100000:0.0:1785220581.441993:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.442004:0:14221:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000c903c40/0xf62499ab927824ee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e3e expref: 442 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.442013:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912192 00000100:00100000:1.0:1785220581.442015:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510607872:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.442016:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912192 00010000:00010000:1.0:1785220581.442028:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.442030:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e62fcd50/0xf62499ab92785ea0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e0 expref: 387 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442038:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442045:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442053:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.442054:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100040 00000100:00100000:0.0:1785220581.442059:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+387:2407:x1871938519912192:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.442061:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.442067:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.442070:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e62fcd50/0xf62499ab92785ea0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e0 expref: 387 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442074:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:0.0:1785220581.442080:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e62fcd50/0xf62499ab92785ea0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e0 expref: 387 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.442189:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1015:3653:x1871938510607232:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13602us (14244us total) trans 4295069865 rc 0/0 00000100:00100000:1.0:1785220581.442197:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284108 00000004:00080000:1.0:1785220581.442245:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725500/2, rc 0, transno 4295069865 00000100:00100000:1.0:1785220581.442257:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510607232:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.442270:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72ea00/2, rc 0, transno 4295070466 00000100:00100000:1.0:1785220581.442279:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510605824:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.442343:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510609792:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.442374:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.442409:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510609792 00000100:00100000:1.0:1785220581.442413:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510609792 00000100:00100000:1.0:1785220581.442436:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284117 00000100:00100000:1.0:1785220581.442443:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1094:3655:x1871938510609792:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.442462:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29165 count 1 00010000:00010000:1.0:1785220581.442481:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442494:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000001a24f4b/0xf62499ab9278651b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac1a expref: 387 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442500:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ed:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.442502:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.442506:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442526:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000001a24f4b/0xf62499ab9278651b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac1a expref: 387 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442538:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000001a24f4b/0xf62499ab9278651b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac1a expref: 387 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.442549:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510609920:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.442581:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01300 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.442624:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.442632:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510610048:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.442655:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.442679:0:14205:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510610048 00000100:00100000:1.0:1785220581.442682:0:14205:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510610048 00000100:00100000:1.0:1785220581.442701:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284118 00000100:00100000:1.0:1785220581.442706:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1015:3653:x1871938510610048:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.442717:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29103 count 1 00010000:00010000:1.0:1785220581.442729:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442740:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000411d4071/0xf62499ab92782c1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79171 expref: 441 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442747:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71af:0x0].0x0 since=33s v=(0 0) 00010000:00010000:1.0:1785220581.442750:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.442754:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442788:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000411d4071/0xf62499ab92782c1f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79171 expref: 441 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.442799:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000411d4071/0xf62499ab92782c1f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79171 expref: 441 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.442809:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510610176:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.442827:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01b40 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.442865:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72e680/2, rc 0, transno 4295070465 00000100:00100000:1.0:1785220581.442874:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510606080:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.442903:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510610304:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.442924:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.442951:0:14204:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510610304 00000100:00100000:1.0:1785220581.442954:0:14204:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510610304 00000100:00100000:1.0:1785220581.442975:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284119 00000100:00100000:1.0:1785220581.442981:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1095:3653:x1871938510610304:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.442992:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29166 count 1 00010000:00010000:1.0:1785220581.443004:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443016:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd504275/0xf62499ab9278664f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac98 expref: 387 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443022:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ee:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.443025:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.443028:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443052:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fd504275/0xf62499ab9278664f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac98 expref: 387 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443061:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fd504275/0xf62499ab9278664f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac98 expref: 387 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.443070:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510610432:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.443110:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.443136:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72f100/2, rc 0, transno 4295070467 00000100:00100000:1.0:1785220581.443145:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510605952:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.443173:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.443181:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510610560:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.443204:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.443227:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510610560 00000100:00100000:1.0:1785220581.443231:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510610560 00000100:00100000:1.0:1785220581.443255:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284120 00000100:00100000:1.0:1785220581.443261:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1096:3655:x1871938510610560:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.443296:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29167 count 1 00010000:00010000:1.0:1785220581.443308:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443320:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000539ad756/0xf62499ab92786759 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad1d expref: 387 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443332:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ef:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.443335:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.443339:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443358:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000539ad756/0xf62499ab92786759 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad1d expref: 387 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.443368:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000539ad756/0xf62499ab92786759 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad1d expref: 387 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.443378:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510610688:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.443397:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.443487:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+387:2407:x1871938519911424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 110us (955us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.443497:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100037 00010000:00010000:2.0:1785220581.443524:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.443533:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.443542:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.443549:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.443561:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1785220581.443690:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d935a2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1097:3654:x1871938510605440:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19602us (20872us total) trans 4295070468 rc 0/0 00000100:00100000:2.0:1785220581.443697:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284101 00010000:00010000:0.0:1785220581.443745:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e62fcd50/0xf62499ab92785ea0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e0 expref: 386 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.443761:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.443775:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.443780:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.443788:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.443809:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.443825:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+385:2407:x1871938519912192:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1765us (1832us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.443831:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100040 00010000:00010000:0.0:1785220581.443864:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.443871:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.443879:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.443887:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.443895:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:0.0:1785220581.443972:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1097:3654:x1871938510605568:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19825us (21008us total) trans 4295070469 rc 0/0 00000100:00100000:0.0:1785220581.443979:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284102 00010000:00010000:0.0:1785220581.445359:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:0.0:1785220581.445370:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445382:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.445386:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445397:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445415:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445423:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a476c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445440:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:0.0:1785220581.445445:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445452:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.445455:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445468:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445478:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445485:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a475e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445499:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220581.445503:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445509:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.445513:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445521:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445530:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445538:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4765 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445553:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.445557:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445563:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.445566:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445574:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445585:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445592:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a477a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445606:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.445610:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445617:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.445620:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445632:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445642:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445648:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4750 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445662:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.445665:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445672:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.445676:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445683:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445691:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445698:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4749 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.445715:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.445716:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.445718:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.445725:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.446171:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72d880/2, rc 0, transno 4295070469 00000100:00100000:1.0:1785220581.446185:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510605568:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.446245:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510610816:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.446294:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.446339:0:14261:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510610816 00000100:00100000:1.0:1785220581.446344:0:14261:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510610816 00000100:00100000:0.0:1785220581.446346:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510606720:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.446366:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.446381:0:14261:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284121 00000100:00100000:1.0:1785220581.446388:0:14261:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1097:3655:x1871938510610816:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.446411:0:14261:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29168 count 1 00010000:00010000:1.0:1785220581.446431:0:14261:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.446447:0:14261:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc35a377/0xf62499ab92786801 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad78 expref: 384 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.446455:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f0:0x0].0x0 since=29s v=(0 0) 00010000:00010000:1.0:1785220581.446459:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.446463:0:14261:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.446488:0:14261:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cc35a377/0xf62499ab92786801 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad78 expref: 384 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.446501:0:14261:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cc35a377/0xf62499ab92786801 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad78 expref: 384 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.446512:0:14261:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0000_UUID:14261:1871938510610944:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.446561:0:14261:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01c00 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.446584:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72fb80/2, rc 0, transno 4295070468 00000100:00100000:1.0:1785220581.446593:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510605440:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.446631:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.446639:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510611072:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.446665:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.446691:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510611072 00000100:00100000:1.0:1785220581.446694:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510611072 00000100:00100000:1.0:1785220581.446726:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284122 00000100:00100000:1.0:1785220581.446734:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1098:3653:x1871938510611072:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.446745:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29169 count 1 00010000:00010000:1.0:1785220581.446758:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.446847:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.446884:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912320 00000100:00100000:2.0:1785220581.446888:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912320 00010000:00010000:1.0:1785220581.446915:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000048183b51/0xf62499ab92786951 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae12 expref: 385 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.446922:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f1:0x0].0x0 since=29s v=(0 0) 00010000:00010000:1.0:1785220581.446926:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.446930:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.446951:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000048183b51/0xf62499ab92786951 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae12 expref: 385 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.446960:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000048183b51/0xf62499ab92786951 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae12 expref: 385 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.446969:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510611200:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.446998:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01900 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.447020:0:14187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000073b09e1c/0xf62499ab92782787 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78f64 expref: 441 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.447039:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510607744:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.447054:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447060:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f0941834/0xf62499ab92786297 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aae6 expref: 385 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447064:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447071:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447079:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.447111:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100041 00000100:00100000:2.0:1785220581.447119:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+385:2407:x1871938519912320:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.447128:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.447132:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f0941834/0xf62499ab92786297 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aae6 expref: 385 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447143:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f0941834/0xf62499ab92786297 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aae6 expref: 385 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447161:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0941834/0xf62499ab92786297 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aae6 expref: 385 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447172:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.447176:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.447181:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447190:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447208:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447209:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.447232:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+384:2407:x1871938519912320:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.447240:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100041 00010000:00010000:2.0:1785220581.447264:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447273:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447281:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.447289:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447298:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.447301:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.447310:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.447314:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447321:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.447325:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447337:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447349:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447355:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a4773 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447370:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.447374:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447379:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ada00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.447382:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447389:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447397:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447404:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4757 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.447422:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.447424:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.447426:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.447431:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1099:3653:x1871938510606208:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19978us (23441us total) trans 4295070470 rc 0/0 00010000:00010000:3.0:1785220581.447432:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.447438:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.447438:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284104 00010000:00010000:2.0:1785220581.447456:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.447528:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1016:3653:x1871938510607616:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19315us (19477us total) trans 4295069866 rc 0/0 00000100:00100000:3.0:1785220581.447535:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284107 00000004:00080000:1.0:1785220581.448852:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f726d80/2, rc 0, transno 4295069866 00000100:00100000:1.0:1785220581.448879:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510607616:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.448893:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72ca80/2, rc 0, transno 4295070470 00000100:00100000:1.0:1785220581.448902:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510606208:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.448950:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510611328:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.448993:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.449039:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510611328 00000100:00100000:0.0:1785220581.449045:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510611328 00000100:00100000:1.0:1785220581.449048:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.449055:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510611456:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.449077:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.449206:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510611456 00000100:00100000:0.0:1785220581.449210:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510611456 00000100:00100000:3.0:1785220581.449320:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284123 00000100:00100000:3.0:1785220581.449332:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1016:3653:x1871938510611328:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.449357:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29104 count 1 00010000:00010000:3.0:1785220581.449378:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.449396:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003c6a5c27/0xf62499ab92782cc7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791d3 expref: 440 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.449404:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b0:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.449409:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.449413:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.449437:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003c6a5c27/0xf62499ab92782cc7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791d3 expref: 440 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.449453:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003c6a5c27/0xf62499ab92782cc7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791d3 expref: 440 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.449464:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510611584:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.449514:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284124 00000100:00100000:0.0:1785220581.449520:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1099:3655:x1871938510611456:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.449530:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29170 count 1 00010000:00010000:0.0:1785220581.449544:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.449556:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006f9b8950/0xf62499ab92786a62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae90 expref: 383 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.449562:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f2:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.449566:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.449570:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.449591:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006f9b8950/0xf62499ab92786a62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae90 expref: 383 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.449601:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006f9b8950/0xf62499ab92786a62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae90 expref: 383 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.449610:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510611712:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.449712:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.449758:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.458972:0:14169:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510608640:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.458982:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.459002:0:14169:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.459006:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912448 00000100:00100000:0.0:1785220581.459011:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912448 00000100:00100000:3.0:1785220581.459031:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510609152:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.459043:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459046:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004cee371a/0xf62499ab9278287c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78fb1 expref: 441 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.459079:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100042 00000100:00100000:0.0:1785220581.459288:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+441:2406:x1871938519912448:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.459304:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.459311:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004cee371a/0xf62499ab9278287c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78fb1 expref: 441 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459322:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004cee371a/0xf62499ab9278287c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78fb1 expref: 441 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459343:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cee371a/0xf62499ab9278287c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78fb1 expref: 441 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459358:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.459362:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.459368:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459378:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459398:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.459429:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510609664:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.459442:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+440:2406:x1871938519912448:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 356us (460us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.459446:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.459450:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100042 00000100:00100000:0.0:1785220581.459491:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.459510:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912704 00000100:00100000:0.0:1785220581.459514:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912704 00010000:00010000:0.0:1785220581.459527:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000988e10ec/0xf62499ab92782668 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ee6 expref: 440 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.459545:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100043 00000100:00100000:0.0:1785220581.459551:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9acc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+440:2406:x1871938519912704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.459558:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.459562:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000988e10ec/0xf62499ab92782668 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ee6 expref: 440 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459578:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000988e10ec/0xf62499ab92782668 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ee6 expref: 440 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459591:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000988e10ec/0xf62499ab92782668 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78ee6 expref: 440 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459601:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.459603:0:14169:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459603:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.459608:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.459612:0:14169:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459616:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.459623:0:14169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.459631:0:14169:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.459641:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.459647:0:14169:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.459658:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9acc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+439:2406:x1871938519912704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (167us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.459664:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100043 00000100:00100000:0.0:1785220581.459752:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.459823:0:14169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1017:3652:x1871938510608000:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24861us (25373us total) trans 4295069867 rc 0/0 00000100:00100000:3.0:1785220581.459831:0:14169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284110 00010000:00010000:3.0:1785220581.459854:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.459860:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.459868:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.459872:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.459881:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.459894:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.459969:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.459978:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.459987:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.459994:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.460011:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.460118:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1017:3655:x1871938510607488:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22286us (23633us total) trans 4295069868 rc 0/0 00000100:00100000:2.0:1785220581.460126:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284112 00010000:00010000:2.0:1785220581.460145:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.460150:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.460157:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.460160:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.460172:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.460184:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.460191:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4796 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.460211:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.460213:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.460215:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.460538:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510608512:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.460556:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.460580:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912576 00000100:00100000:1.0:1785220581.460584:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912576 00010000:00010000:1.0:1785220581.460614:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f86d3395/0xf62499ab92782b31 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7910f expref: 439 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.460646:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100044 00000100:00100000:1.0:1785220581.460653:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+439:2406:x1871938519912576:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.460662:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.460667:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f86d3395/0xf62499ab92782b31 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7910f expref: 439 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.460676:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f86d3395/0xf62499ab92782b31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7910f expref: 439 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.460689:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f86d3395/0xf62499ab92782b31 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7910f expref: 439 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.460705:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.460708:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.460713:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.460721:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.460737:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.460758:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+438:2406:x1871938519912576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (1006us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.460775:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100044 00010000:00010000:1.0:1785220581.460794:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.460802:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.460811:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.460925:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4788 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.460944:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.460945:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.460947:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.460956:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.460995:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912832 00000100:00100000:2.0:1785220581.461000:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912832 00010000:00010000:2.0:1785220581.461016:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000054875f6c/0xf62499ab92782557 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e84 expref: 438 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.461048:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100045 00000100:00100000:2.0:1785220581.461056:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+438:2406:x1871938519912832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.461065:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.461069:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000054875f6c/0xf62499ab92782557 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e84 expref: 438 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461079:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000054875f6c/0xf62499ab92782557 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e84 expref: 438 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461132:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054875f6c/0xf62499ab92782557 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78e84 expref: 438 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461142:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.461145:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.461149:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461156:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461173:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.461194:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+437:2406:x1871938519912832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 139us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.461201:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100045 00010000:00010000:2.0:1785220581.461223:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461232:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461239:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461246:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461259:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.461373:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1017:3654:x1871938510607360:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22794us (24724us total) trans 4295069869 rc 0/0 00000100:00100000:2.0:1785220581.461380:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284116 00010000:00010000:1.0:1785220581.461461:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.461474:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.461713:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.461718:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461726:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffa00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.461730:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461738:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461750:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461758:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a47b2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461786:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.461790:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461801:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.461804:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461811:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461822:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461829:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4781 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.461855:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.461858:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.461860:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.462319:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1017:3652:x1871938510608384:12345-0@lo:6:osp-syn-1-0.0 Request processed in 27669us (27928us total) trans 4295069870 rc 0/0 00000100:00100000:1.0:1785220581.462328:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284109 00000004:00080000:1.0:1785220581.462376:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93664000/2, rc 0, transno 4295069867 00000100:00100000:1.0:1785220581.462388:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510608000:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.462397:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93666680/2, rc 0, transno 4295069870 00000100:00100000:1.0:1785220581.462400:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510608384:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.462425:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f727800/2, rc 0, transno 4295069868 00000100:00100000:1.0:1785220581.462428:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510607488:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.462495:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.462505:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510611968:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.462539:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.462568:0:13818:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510611968 00000100:00100000:1.0:1785220581.462573:0:13818:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510611968 00000100:00100000:1.0:1785220581.462603:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284125 00000100:00100000:1.0:1785220581.462610:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1017:3653:x1871938510611968:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.462630:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29106 count 1 00010000:00010000:1.0:1785220581.462648:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.462663:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7e5e7cf/0xf62499ab92782e95 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792ac expref: 436 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.462671:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b2:0x0].0x0 since=32s v=(0 0) 00010000:00010000:1.0:1785220581.462675:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.462679:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.462701:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e7e5e7cf/0xf62499ab92782e95 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792ac expref: 436 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.462713:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e7e5e7cf/0xf62499ab92782e95 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792ac expref: 436 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.462724:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510612096:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.462782:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.462797:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510611840:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.462819:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.463024:0:14185:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510611840 00000100:00100000:1.0:1785220581.463028:0:14185:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510611840 00000100:00100000:1.0:1785220581.463049:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284126 00000100:00100000:1.0:1785220581.463054:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1018:3655:x1871938510611840:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.463082:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29105 count 1 00010000:00010000:1.0:1785220581.463124:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463203:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004a50c305/0xf62499ab92782d84 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79227 expref: 436 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463281:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b1:0x0].0x0 since=32s v=(0 0) 00010000:00010000:1.0:1785220581.463285:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.463289:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463313:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004a50c305/0xf62499ab92782d84 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79227 expref: 436 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463322:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004a50c305/0xf62499ab92782d84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79227 expref: 436 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.463332:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510612224:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.463359:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01d80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.463398:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510612352:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.463423:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.463482:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f727100/2, rc 0, transno 4295069869 00000100:00100000:1.0:1785220581.463490:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510607360:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.463519:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.463525:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9147b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510612480:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.463535:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510609280:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.463546:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.463556:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.463580:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510612352 00000100:00100000:3.0:1785220581.463584:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510612352 00000100:00100000:3.0:1785220581.463614:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510612480 00000100:00100000:3.0:1785220581.463618:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510612480 00000100:00100000:3.0:1785220581.463638:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284127 00000100:00100000:3.0:1785220581.463644:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1020:3653:x1871938510612352:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.463657:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29107 count 1 00010000:00010000:3.0:1785220581.463670:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.463670:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284128 00000100:00100000:1.0:1785220581.463677:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1020:3655:x1871938510612480:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.463681:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000050083d22/0xf62499ab92782f28 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7932a expref: 436 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.463688:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b3:0x0].0x0 since=32s v=(0 0) 00002000:00080000:1.0:1785220581.463688:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29108 count 1 00010000:00010000:3.0:1785220581.463691:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.463695:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463697:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463707:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000573b99d/0xf62499ab9278300f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79377 expref: 436 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.463713:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000050083d22/0xf62499ab92782f28 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7932a expref: 436 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463721:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b4:0x0].0x0 since=32s v=(0 0) 00010000:00010000:3.0:1785220581.463723:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000050083d22/0xf62499ab92782f28 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7932a expref: 436 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463724:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.463728:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.463740:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0001_UUID:14192:1871938510612608:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.463744:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000573b99d/0xf62499ab9278300f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79377 expref: 436 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.463751:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000573b99d/0xf62499ab9278300f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79377 expref: 436 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.463760:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510612736:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.463774:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.463791:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.463794:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.463808:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510609408:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.463822:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.466493:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.466524:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.466552:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519912960 00000100:00100000:1.0:1785220581.466562:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519912960 00000100:00100000:2.0:1785220581.466595:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510609024:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.466602:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007f0e66be/0xf62499ab92782a3c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7907c expref: 437 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.466619:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.466637:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913216 00000100:00100000:1.0:1785220581.466640:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913216 00000100:00100000:2.0:1785220581.466649:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.466655:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ec27ae0/0xf62499ab9278648f lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abd4 expref: 384 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.466675:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913088 00000100:00100000:1.0:1785220581.466678:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913088 00010000:00010000:1.0:1785220581.466690:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006b91fab2/0xf62499ab92782939 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7901a expref: 438 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.466717:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100046 00000100:00100000:1.0:1785220581.466726:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+438:2406:x1871938519912960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.466739:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.466745:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007f0e66be/0xf62499ab92782a3c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7907c expref: 438 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.466756:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007f0e66be/0xf62499ab92782a3c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7907c expref: 438 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.466792:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f0e66be/0xf62499ab92782a3c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7907c expref: 438 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.466808:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.466812:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.466818:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.466827:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.466860:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.466903:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+437:2406:x1871938519912960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 178us (419us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.466910:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.466920:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.466929:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.466936:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.466953:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.467151:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1021:3655:x1871938510608256:12345-0@lo:6:osp-syn-1-0.0 Request processed in 29158us (30615us total) trans 4295069871 rc 0/0 00000100:00100000:3.0:1785220581.467160:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284113 00000100:00100000:3.0:1785220581.467195:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510609536:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.467208:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.467663:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100047 00000100:00100000:2.0:1785220581.467675:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+384:2406:x1871938519913216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.467686:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.467691:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ec27ae0/0xf62499ab9278648f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abd4 expref: 384 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.467702:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ec27ae0/0xf62499ab9278648f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abd4 expref: 384 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.467718:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ec27ae0/0xf62499ab9278648f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abd4 expref: 384 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.467730:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.467733:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.467738:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.467746:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.467779:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.467805:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2406:x1871938519913216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (1280us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.467819:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100046 00000100:00100000:1.0:1785220581.467866:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100048 00000100:00100000:1.0:1785220581.467875:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+436:2407:x1871938519913088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.467883:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.467889:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006b91fab2/0xf62499ab92782939 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7901a expref: 436 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.467898:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006b91fab2/0xf62499ab92782939 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7901a expref: 436 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.467911:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b91fab2/0xf62499ab92782939 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7901a expref: 436 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.467921:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.467924:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.467928:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.467936:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.467944:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100047 00010000:00010000:2.0:1785220581.467965:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.467971:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.467985:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.467989:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.467999:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.468003:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.468011:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.468018:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a479d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.468022:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+435:2407:x1871938519913088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 149us (1372us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.468028:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100048 00010000:00010000:2.0:1785220581.468039:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.468041:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.468043:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.468050:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.468058:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.468062:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510610432:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.468066:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.468077:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.468078:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468174:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468184:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468192:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468200:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468213:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.468282:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510610688:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.468297:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.468317:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1021:3655:x1871938510608128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 30180us (31735us total) trans 4295069872 rc 0/0 00000100:00100000:1.0:1785220581.468321:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510609920:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.468325:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284114 00010000:00010000:1.0:1785220581.468330:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468343:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.468347:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.468351:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510610176:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.468354:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.468357:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.468361:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468372:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468383:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468389:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a47a4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.468392:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93665180/2, rc 0, transno 4295069872 00000100:00100000:1.0:1785220581.468402:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510608128:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.468404:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.468406:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.468407:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.468412:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93667100/2, rc 0, transno 4295069871 00000100:00100000:1.0:1785220581.468415:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510608256:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.468475:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.468496:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.468509:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510612992:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.468541:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.468560:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1100:3655:x1871938510608896:12345-0@lo:6:osp-syn-0-0.0 Request processed in 30289us (31940us total) trans 4295070471 rc 0/0 00000100:00100000:2.0:1785220581.468567:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284115 00000100:00100000:1.0:1785220581.468578:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510612864:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.468580:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510612992 00000100:00100000:2.0:1785220581.468585:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510612992 00000100:00100000:1.0:1785220581.468601:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.468630:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.468636:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468644:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.468647:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468654:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468664:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468677:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a47ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.468711:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.468713:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.468713:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724a80/2, rc 0, transno 4295070471 00010000:00010000:3.0:1785220581.468715:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.468722:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510608896:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.468741:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510612864 00000100:00100000:2.0:1785220581.468745:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510612864 00000100:00100000:1.0:1785220581.468774:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510613120:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.468798:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.468808:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510613120 00000100:00100000:2.0:1785220581.468811:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510613120 00000100:00100000:2.0:1785220581.468831:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284129 00000100:00100000:2.0:1785220581.468837:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1022:3653:x1871938510612992:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.468859:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29110 count 1 00010000:00010000:2.0:1785220581.468878:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.468893:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d8983b33/0xf62499ab92783238 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79465 expref: 434 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.468900:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b6:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.468903:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.468907:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.468916:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284130 00000100:00100000:0.0:1785220581.468925:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9573cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1022:3655:x1871938510612864:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.468934:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d8983b33/0xf62499ab92783238 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79465 expref: 434 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220581.468937:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29109 count 1 00010000:00010000:2.0:1785220581.468946:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d8983b33/0xf62499ab92783238 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79465 expref: 434 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.468948:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.468959:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510613248:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.468960:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003072b4e6/0xf62499ab92783158 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793fc expref: 434 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.468972:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b5:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.468975:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.468978:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.468995:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003072b4e6/0xf62499ab92783158 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793fc expref: 434 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469002:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.469004:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003072b4e6/0xf62499ab92783158 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793fc expref: 434 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.469014:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510613376:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.469016:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284131 00000100:00100000:2.0:1785220581.469023:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1100:3653:x1871938510613120:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.469034:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29171 count 1 00000100:00100000:0.0:1785220581.469036:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd596c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.469047:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469058:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b892444f/0xf62499ab92786b03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aef2 expref: 382 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469064:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f3:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.469067:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.469070:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.469398:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.469431:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.469445:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b892444f/0xf62499ab92786b03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aef2 expref: 382 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.469448:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913344 00000100:00100000:0.0:1785220581.469451:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913344 00010000:00010000:2.0:1785220581.469461:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b892444f/0xf62499ab92786b03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aef2 expref: 383 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.469467:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000227071a4/0xf62499ab9278641f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab80 expref: 383 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469472:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510613504:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.469486:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913600 00000100:00100000:0.0:1785220581.469489:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913600 00000100:00100000:2.0:1785220581.469501:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.469502:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000411d4071/0xf62499ab92782c1f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79171 expref: 435 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.469519:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100049 00000100:00100000:0.0:1785220581.469526:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2407:x1871938519913344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.469535:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.469541:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.469541:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000227071a4/0xf62499ab9278641f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab80 expref: 383 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.469552:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000227071a4/0xf62499ab9278641f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab80 expref: 383 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469559:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.469567:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000227071a4/0xf62499ab9278641f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab80 expref: 383 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469575:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.469578:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.469581:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.469586:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469588:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913984 00000100:00100000:2.0:1785220581.469591:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913984 00010000:00010000:0.0:1785220581.469595:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469604:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cc35a377/0xf62499ab92786801 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad78 expref: 383 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.469613:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.469621:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913472 00000100:00100000:2.0:1785220581.469623:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913472 00000100:00100000:0.0:1785220581.469632:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+384:2407:x1871938519913344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (235us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.469634:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000001a24f4b/0xf62499ab9278651b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac1a expref: 384 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.469643:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100049 00000100:00100000:0.0:1785220581.469654:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913728 00000100:00100000:0.0:1785220581.469656:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913728 00010000:00010000:0.0:1785220581.469667:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fd504275/0xf62499ab9278664f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac98 expref: 384 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.469685:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510611584:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.469702:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0000_UUID:14261:1871938510610944:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.469703:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.469717:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.469801:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100050 00000100:00100000:1.0:1785220581.469809:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90703050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+435:2407:x1871938519913600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.469817:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.469822:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000411d4071/0xf62499ab92782c1f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79171 expref: 435 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469825:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100051 00000100:00100000:2.0:1785220581.469831:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+384:2407:x1871938519913984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.469839:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.469843:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cc35a377/0xf62499ab92786801 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad78 expref: 384 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469858:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cc35a377/0xf62499ab92786801 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad78 expref: 384 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469871:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc35a377/0xf62499ab92786801 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad78 expref: 384 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469881:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.469884:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.469889:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469896:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.469898:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000411d4071/0xf62499ab92782c1f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79171 expref: 435 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.469914:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000411d4071/0xf62499ab92782c1f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79171 expref: 435 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469915:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.469924:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.469927:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.469931:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469932:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2407:x1871938519913984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.469939:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100051 00010000:00010000:1.0:1785220581.469940:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.469955:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100052 00010000:00010000:1.0:1785220581.469955:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.469961:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+382:2407:x1871938519913472:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.469968:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220581.469972:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90703050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+434:2407:x1871938519913600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 165us (541us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.469973:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000001a24f4b/0xf62499ab9278651b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac1a expref: 382 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.469978:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100050 00010000:00010000:2.0:1785220581.469987:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000001a24f4b/0xf62499ab9278651b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac1a expref: 382 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.469998:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a24f4b/0xf62499ab9278651b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac1a expref: 382 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.470007:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.470009:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ed:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.470011:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100053 00010000:00010000:2.0:1785220581.470014:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.470017:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+381:2407:x1871938519913728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.470022:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.470023:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.470028:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fd504275/0xf62499ab9278664f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac98 expref: 381 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.470037:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.470037:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fd504275/0xf62499ab9278664f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac98 expref: 381 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.470049:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd504275/0xf62499ab9278664f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac98 expref: 381 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.470053:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+381:2407:x1871938519913472:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (493us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.470059:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100052 00010000:00010000:1.0:1785220581.470059:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.470062:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.470066:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.470078:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.470283:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.470307:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+379:2407:x1871938519913728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 290us (730us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.470314:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100053 00010000:00010000:0.0:1785220581.470376:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.470386:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.470395:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.470403:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.470411:0:14261:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.470418:0:14261:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.470425:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.470427:0:14261:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.470433:0:14261:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.470445:0:14261:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.470528:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.470534:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.470542:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.470545:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.470554:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.470561:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1102:3653:x1871938510608768:12345-0@lo:6:osp-syn-0-0.0 Request processed in 33124us (34168us total) trans 4295070473 rc 0/0 00010000:00010000:3.0:1785220581.470565:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.470568:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284111 00010000:00010000:3.0:1785220581.470572:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005275d713/0xf62499ab927a478f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.470587:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.470590:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.470596:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.470599:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.470606:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.470615:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.470622:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a47d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.470626:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.470635:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.470637:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.470639:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.470681:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.470703:0:14261:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1101:3655:x1871938510610816:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24315us (24409us total) trans 4295070472 rc 0/0 00000100:00100000:2.0:1785220581.470711:0:14261:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284121 00010000:00010000:2.0:1785220581.470729:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.470737:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519914112 00000100:00100000:2.0:1785220581.470742:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519914112 00000100:00100000:0.0:1785220581.470745:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.470757:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003c6a5c27/0xf62499ab92782cc7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791d3 expref: 434 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.470790:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.470806:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510611712:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.470806:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519914240 00000100:00100000:0.0:1785220581.470809:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519914240 00010000:00010000:3.0:1785220581.470821:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.470823:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000048183b51/0xf62499ab92786951 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae12 expref: 379 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.470931:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519913856 00000100:00100000:0.0:1785220581.470935:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519913856 00010000:00010000:0.0:1785220581.470950:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000539ad756/0xf62499ab92786759 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad1d expref: 380 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.470964:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519914368 00000100:00100000:2.0:1785220581.470968:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519914368 00000100:00100000:0.0:1785220581.470973:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100054 00000100:00100000:0.0:1785220581.470980:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+381:2407:x1871938519914240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.470981:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006f9b8950/0xf62499ab92786a62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae90 expref: 381 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.470989:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.470994:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000048183b51/0xf62499ab92786951 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae12 expref: 381 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.470999:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100055 00000100:00100000:2.0:1785220581.471005:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+434:2407:x1871938519914112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.471005:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000048183b51/0xf62499ab92786951 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae12 expref: 381 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471012:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.471015:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003c6a5c27/0xf62499ab92782cc7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791d3 expref: 434 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.471019:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.471022:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.471025:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003c6a5c27/0xf62499ab92782cc7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791d3 expref: 434 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.471027:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.471036:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471038:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c6a5c27/0xf62499ab92782cc7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b791d3 expref: 434 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471049:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.471052:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.471053:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.471055:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471063:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.471070:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+381:2407:x1871938519914240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (389us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.471076:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100054 00010000:00010000:2.0:1785220581.471077:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.471110:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100056 00000100:00100000:0.0:1785220581.471117:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+380:2406:x1871938519913856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.471124:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.471129:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000539ad756/0xf62499ab92786759 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad1d expref: 380 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.471150:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000539ad756/0xf62499ab92786759 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad1d expref: 380 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.471162:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000539ad756/0xf62499ab92786759 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad1d expref: 380 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.471174:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.471177:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.471180:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.471187:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+433:2407:x1871938519914112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 182us (560us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.471189:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.471194:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100055 00000100:00100000:2.0:1785220581.471209:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100057 00010000:00010000:0.0:1785220581.471210:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.471214:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce71c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+379:2406:x1871938519914368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.471222:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.471226:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006f9b8950/0xf62499ab92786a62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae90 expref: 379 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471236:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006f9b8950/0xf62499ab92786a62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae90 expref: 379 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471253:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006f9b8950/0xf62499ab92786a62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae90 expref: 379 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471263:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.471266:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.471270:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471278:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471294:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.471313:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce71c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+378:2406:x1871938519914368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (522us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.471319:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100057 00000100:00100000:2.0:1785220581.471369:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.471382:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519914496 00000100:00100000:2.0:1785220581.471385:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519914496 00010000:00010000:2.0:1785220581.471397:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e7e5e7cf/0xf62499ab92782e95 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792ac expref: 433 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.471416:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100058 00000100:00100000:2.0:1785220581.471422:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+433:2406:x1871938519914496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.471429:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.471434:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e7e5e7cf/0xf62499ab92782e95 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792ac expref: 433 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471443:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e7e5e7cf/0xf62499ab92782e95 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792ac expref: 433 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471456:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.471459:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.471463:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471471:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.471482:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.471483:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+377:2406:x1871938519913856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 367us (737us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.471490:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100056 00000100:00100000:2.0:1785220581.471495:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+433:2406:x1871938519914496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (127us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.471501:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100058 00000100:00100000:2.0:1785220581.471647:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.471708:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519914624 00000100:00100000:3.0:1785220581.471713:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519914624 00010000:00010000:3.0:1785220581.471731:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004a50c305/0xf62499ab92782d84 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79227 expref: 433 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.471789:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.471850:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519914752 00000100:00100000:3.0:1785220581.471854:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519914752 00010000:00010000:3.0:1785220581.471870:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000573b99d/0xf62499ab9278300f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79377 expref: 434 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.471960:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100059 00000100:00100000:3.0:1785220581.471968:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+434:2407:x1871938519914624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.471976:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.471981:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004a50c305/0xf62499ab92782d84 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79227 expref: 434 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.471991:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004a50c305/0xf62499ab92782d84 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79227 expref: 434 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472005:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.472008:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.472012:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472019:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472033:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.472059:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+434:2407:x1871938519914624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (412us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.472065:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100059 00000100:00100000:3.0:1785220581.472082:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100060 00000100:00100000:3.0:1785220581.472115:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+433:2406:x1871938519914752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.472143:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.472149:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000573b99d/0xf62499ab9278300f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79377 expref: 433 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472158:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000573b99d/0xf62499ab9278300f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79377 expref: 433 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472169:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.472171:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.472175:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472182:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472193:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.472210:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+433:2406:x1871938519914752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.472216:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100060 00000100:00100000:3.0:1785220581.472250:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0001_UUID:14192:1871938510612608:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.472265:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472283:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472290:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472300:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472308:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472323:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.472465:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472475:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472483:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472490:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.472499:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1023:3653:x1871938510611328:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23170us (23505us total) trans 4295069873 rc 0/0 00000100:00100000:3.0:1785220581.472506:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284123 00010000:00010000:1.0:1785220581.472506:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.472529:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472537:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472545:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472557:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472569:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220581.472593:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1101:3655:x1871938510610560:12345-0@lo:6:osp-syn-0-0.0 Request processed in 29331us (29387us total) trans 4295070474 rc 0/0 00000100:00100000:1.0:1785220581.472599:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284120 00010000:00010000:1.0:1785220581.472625:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472639:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.472641:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1101:3655:x1871938510611456:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23121us (23562us total) trans 4295070475 rc 0/0 00000100:00100000:3.0:1785220581.472647:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284124 00010000:00010000:1.0:1785220581.472648:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472655:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472662:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.472668:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:3.0:1785220581.472675:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472682:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.472686:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472695:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472706:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472714:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a47e3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.472731:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1101:3655:x1871938510609792:12345-0@lo:6:osp-syn-0-0.0 Request processed in 30288us (30356us total) trans 4295070476 rc 0/0 00010000:00010000:3.0:1785220581.472732:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.472736:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.472737:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284117 00010000:00010000:3.0:1785220581.472742:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.472745:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472753:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472757:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472780:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472787:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472795:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472795:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472803:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a47ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.472812:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:3.0:1785220581.472817:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.472820:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472826:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.472829:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472836:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472851:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472858:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a47ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472871:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.472874:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472881:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.472884:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472892:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472901:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472908:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a47b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472921:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.472924:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472931:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.472935:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.472935:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1101:3653:x1871938510610304:12345-0@lo:6:osp-syn-0-0.0 Request processed in 29955us (30010us total) trans 4295070477 rc 0/0 00000100:00100000:1.0:1785220581.472942:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284119 00010000:00010000:3.0:1785220581.472943:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472953:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472959:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a47c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472971:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.472972:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.472972:0:14185:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a50c305/0xf62499ab92782d84 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79227 expref: 432 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.472974:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.472980:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.472988:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510612224:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.473000:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473008:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473014:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473022:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473028:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473050:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.473373:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1023:3655:x1871938510611840:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10316us (10551us total) trans 4295069874 rc 0/0 00000100:00100000:1.0:1785220581.473380:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284126 00010000:00010000:1.0:1785220581.473413:0:14203:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000573b99d/0xf62499ab9278300f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79377 expref: 431 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.473426:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510612736:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.473439:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473447:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473454:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473461:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473468:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473479:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.473550:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1023:3655:x1871938510612480:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9873us (10002us total) trans 4295069875 rc 0/0 00000100:00100000:1.0:1785220581.473556:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284128 00010000:00010000:1.0:1785220581.473577:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.473581:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473588:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.473591:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473599:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473611:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473619:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a47f8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473634:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.473638:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473644:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.473647:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473654:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473666:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473674:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4806 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473685:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.473687:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.473688:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.473709:0:13836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048183b51/0xf62499ab92786951 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae12 expref: 376 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.473719:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510611200:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.473731:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473739:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473745:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473753:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473781:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473795:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.473882:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1101:3653:x1871938510611072:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27149us (27216us total) trans 4295070478 rc 0/0 00000100:00100000:1.0:1785220581.473889:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284122 00010000:00010000:1.0:1785220581.473905:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.473909:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473917:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.473921:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473927:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473937:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473944:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a47dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.473958:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.473960:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.473962:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.473982:0:13818:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7e5e7cf/0xf62499ab92782e95 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b792ac expref: 430 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.473994:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510612096:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.474005:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474014:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474020:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474027:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474034:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474045:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.474151:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1023:3653:x1871938510611968:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11541us (11611us total) trans 4295069876 rc 0/0 00000100:00100000:1.0:1785220581.474158:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284125 00010000:00010000:1.0:1785220581.474174:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.474178:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474185:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.474188:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474195:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474206:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474397:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a47f1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474414:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.474416:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.474417:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.474429:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474437:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474446:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474458:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474472:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.474546:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1023:3653:x1871938510610048:12345-0@lo:6:osp-syn-1-0.0 Request processed in 31838us (31889us total) trans 4295069877 rc 0/0 00000100:00100000:1.0:1785220581.474553:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284118 00010000:00010000:1.0:1785220581.474567:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.474568:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.474571:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.474573:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.474578:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.474581:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474589:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474600:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474607:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a47c0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.474619:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.474621:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.474623:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.474654:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915adc00/2, rc 0, transno 4295069876 00000100:00100000:1.0:1785220581.474665:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510611968:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.474676:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93665c00/2, rc 0, transno 4295069873 00000100:00100000:1.0:1785220581.474679:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510611328:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.474689:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725f80/2, rc 0, transno 4295070478 00000100:00100000:1.0:1785220581.474697:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510611072:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.474705:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be5180/2, rc 0, transno 4295070477 00000100:00100000:1.0:1785220581.474709:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510610304:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.474715:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be6300/2, rc 0, transno 4295069877 00000100:00100000:1.0:1785220581.474719:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510610048:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.474726:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724700/2, rc 0, transno 4295070473 00000100:00100000:1.0:1785220581.474729:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510608768:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.475035:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.475048:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510613888:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.475079:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.475146:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510613760:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.475170:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.475194:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9147b800/2, rc 0, transno 4295069875 00000100:00100000:1.0:1785220581.475198:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9147b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510612480:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.475207:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ad180/2, rc 0, transno 4295069874 00000100:00100000:1.0:1785220581.475210:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510611840:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.475956:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915aca80/2, rc 0, transno 4295070475 00000100:00100000:1.0:1785220581.475968:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510611456:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.475978:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be4000/2, rc 0, transno 4295070472 00000100:00100000:1.0:1785220581.475981:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510610816:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.475989:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be4a80/2, rc 0, transno 4295070474 00000100:00100000:1.0:1785220581.475992:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510610560:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.476000:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be7100/2, rc 0, transno 4295070476 00000100:00100000:1.0:1785220581.476003:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510609792:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.476013:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.476020:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510613632:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.476073:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.476193:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614016:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.476295:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.476317:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.476323:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614400:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.476342:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.476360:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510613888 00000100:00100000:2.0:1785220581.476367:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510613888 00000100:00100000:1.0:1785220581.476367:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614272:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.476385:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.476401:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.476402:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510613760 00000100:00100000:2.0:1785220581.476405:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510613760 00000100:00100000:1.0:1785220581.476406:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614144:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.476420:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510613632 00000100:00100000:2.0:1785220581.476423:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510613632 00000100:00100000:1.0:1785220581.476424:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.476440:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614016 00000100:00100000:2.0:1785220581.476443:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614016 00000100:00100000:2.0:1785220581.476457:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614400 00000100:00100000:2.0:1785220581.476460:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614400 00000100:00100000:2.0:1785220581.476475:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614272 00000100:00100000:2.0:1785220581.476477:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614272 00000100:00100000:2.0:1785220581.476493:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614144 00000100:00100000:2.0:1785220581.476495:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614144 00000100:00100000:1.0:1785220581.476503:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90143b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614528:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.476514:0:14170:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284132 00000100:00100000:2.0:1785220581.476522:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b033100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+1107:3653:x1871938510613888:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.476524:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.476539:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510615040:0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.476550:0:14170:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29174 count 1 00000100:00100000:1.0:1785220581.476555:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.476571:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:2.0:1785220581.476575:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.476575:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90140e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614912:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.476595:0:14170:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000443b641a/0xf62499ab92786ddb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b049 expref: 375 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.476604:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f6:0x0].0x0 since=29s v=(0 0) 00000100:00100000:1.0:1785220581.476604:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.476608:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.476613:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.476628:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90143100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614784:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.476639:0:14170:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000443b641a/0xf62499ab92786ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b049 expref: 375 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.476645:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.476652:0:14170:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000443b641a/0xf62499ab92786ddb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b049 expref: 375 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.476660:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.476663:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90142680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614656:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.476664:0:14170:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510615168:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.476966:0:14170:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.476984:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614528 00000100:00100000:2.0:1785220581.476988:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614528 00000100:00100000:2.0:1785220581.477012:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510615040 00000100:00100000:2.0:1785220581.477015:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510615040 00000100:00100000:2.0:1785220581.477030:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284133 00000100:00100000:2.0:1785220581.477037:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1107:3655:x1871938510613760:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.477052:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29173 count 1 00010000:00010000:2.0:1785220581.477064:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.477075:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cdeaf63c/0xf62499ab92786cfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7afe7 expref: 375 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.477082:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f5:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.477105:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.477109:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.477130:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cdeaf63c/0xf62499ab92786cfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7afe7 expref: 375 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.477139:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cdeaf63c/0xf62499ab92786cfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7afe7 expref: 375 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.477150:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510615296:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.477289:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614912 00000100:00100000:1.0:1785220581.477289:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.477293:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614912 00000100:00100000:3.0:1785220581.477315:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614784 00000100:00100000:1.0:1785220581.477316:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510614656 00000100:00100000:3.0:1785220581.477318:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614784 00000100:00100000:1.0:1785220581.477319:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510614656 00000100:00100000:3.0:1785220581.477334:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284134 00000100:00100000:1.0:1785220581.477339:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284135 00000100:00100000:3.0:1785220581.477341:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1107:3655:x1871938510613632:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.477344:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1107:3653:x1871938510614016:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.477353:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29172 count 1 00002000:00080000:1.0:1785220581.477353:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29175 count 1 00010000:00010000:1.0:1785220581.477365:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477366:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.477378:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009b9e14c7/0xf62499ab92786ef3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b0a4 expref: 375 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477379:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d39859b3/0xf62499ab92786bea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af46 expref: 375 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.477385:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f7:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.477386:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f4:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.477389:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.477389:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.477393:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.477393:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477411:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d39859b3/0xf62499ab92786bea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af46 expref: 375 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.477411:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009b9e14c7/0xf62499ab92786ef3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b0a4 expref: 375 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477421:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d39859b3/0xf62499ab92786bea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af46 expref: 375 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.477426:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009b9e14c7/0xf62499ab92786ef3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b0a4 expref: 375 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.477431:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510615424:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.477435:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9363aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510615552:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.477462:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.477462:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.477479:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284136 00000100:00100000:3.0:1785220581.477487:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1107:3653:x1871938510614400:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.477499:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29178 count 1 00010000:00010000:3.0:1785220581.477510:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477521:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000bf99e8b/0xf62499ab927871f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1d1 expref: 375 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477528:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71fa:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.477530:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.477534:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477552:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000bf99e8b/0xf62499ab927871f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1d1 expref: 375 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.477558:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284137 00010000:00010000:3.0:1785220581.477562:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000bf99e8b/0xf62499ab927871f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1d1 expref: 375 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.477568:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1107:3655:x1871938510614272:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.477570:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510615680:192.168.201.51@tcp:104: 00002000:00080000:0.0:1785220581.477580:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29177 count 1 00000100:00100000:3.0:1785220581.477592:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.477592:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.477603:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d25cb02d/0xf62499ab92787169 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b18b expref: 375 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.477609:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284138 00010000:00010000:0.0:1785220581.477609:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f9:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.477612:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f9:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.477616:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1107:3655:x1871938510614144:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.477616:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.477628:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29176 count 1 00010000:00010000:0.0:1785220581.477633:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d25cb02d/0xf62499ab92787169 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b18b expref: 375 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477638:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.477643:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d25cb02d/0xf62499ab92787169 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b18b expref: 375 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477648:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000084dfb94b/0xf62499ab92787027 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b129 expref: 375 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477655:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71f8:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.477658:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f8:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.477659:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510615808:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.477662:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.477677:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000084dfb94b/0xf62499ab92787027 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b129 expref: 375 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.477681:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59000 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.477686:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000084dfb94b/0xf62499ab92787027 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b129 expref: 375 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.477695:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510615936:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.477697:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284139 00000100:00100000:0.0:1785220581.477703:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfa5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1027:3653:x1871938510614528:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.477715:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00002000:00080000:0.0:1785220581.477715:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29111 count 1 00010000:00010000:0.0:1785220581.477728:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.477741:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c18eb213/0xf62499ab92783311 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794b2 expref: 429 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.477747:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b7:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.477750:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.477753:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.477754:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.477782:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284140 00010000:00010000:0.0:1785220581.477784:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c18eb213/0xf62499ab92783311 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794b2 expref: 429 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.477790:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfa5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1027:3653:x1871938510615040:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.477794:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c18eb213/0xf62499ab92783311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794b2 expref: 429 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.477800:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29115 count 1 00000100:00100000:0.0:1785220581.477805:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510616064:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.477810:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.477822:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001e886e8f/0xf62499ab9278378d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796b8 expref: 429 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.477825:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59a80 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.477829:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71bb:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.477833:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.477836:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.477854:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001e886e8f/0xf62499ab9278378d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796b8 expref: 429 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.477863:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001e886e8f/0xf62499ab9278378d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796b8 expref: 429 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.477872:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510616192:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.477892:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.478318:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284141 00000100:00100000:2.0:1785220581.478326:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284142 00000100:00100000:0.0:1785220581.478330:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1027:3653:x1871938510614912:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.478333:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1027:3655:x1871938510614784:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.478344:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29113 count 1 00002000:00080000:0.0:1785220581.478344:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29114 count 1 00010000:00010000:2.0:1785220581.478357:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478357:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.478370:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000016ee4909/0xf62499ab92783510 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b795bc expref: 429 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478370:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001a927c28/0xf62499ab927836bb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7965d expref: 429 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.478378:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b9:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.478378:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ba:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.478382:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.478382:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.478387:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478387:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478405:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001a927c28/0xf62499ab927836bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7965d expref: 429 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.478406:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000016ee4909/0xf62499ab92783510 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b795bc expref: 429 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.478416:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000016ee4909/0xf62499ab92783510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b795bc expref: 429 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478416:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001a927c28/0xf62499ab927836bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7965d expref: 429 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.478427:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510616448:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.478427:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510616320:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.478458:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.478475:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284143 00000100:00100000:0.0:1785220581.478484:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d977f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1027:3655:x1871938510614656:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.478495:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00002000:00080000:0.0:1785220581.478498:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29112 count 1 00010000:00010000:0.0:1785220581.478509:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478519:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006ef00579/0xf62499ab92783406 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79537 expref: 429 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478526:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71b8:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.478528:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.478532:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478554:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006ef00579/0xf62499ab92783406 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79537 expref: 429 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.478563:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006ef00579/0xf62499ab92783406 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79537 expref: 429 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.478574:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510616576:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.478591:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.478825:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.479134:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510613504:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.479157:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.479312:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519914880 00000100:00100000:0.0:1785220581.479317:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519914880 00010000:00010000:0.0:1785220581.479343:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000050083d22/0xf62499ab92782f28 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7932a expref: 430 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.479372:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100061 00000100:00100000:0.0:1785220581.479380:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93638700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+430:2407:x1871938519914880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.479390:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.479396:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000050083d22/0xf62499ab92782f28 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7932a expref: 430 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.479407:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000050083d22/0xf62499ab92782f28 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7932a expref: 430 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.479428:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000050083d22/0xf62499ab92782f28 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7932a expref: 430 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.479443:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.479447:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.479452:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.479461:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.479465:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.479486:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.479590:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510613248:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.479604:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.479784:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.479807:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915136 00000100:00100000:1.0:1785220581.479811:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915136 00010000:00010000:1.0:1785220581.479826:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003072b4e6/0xf62499ab92783158 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793fc expref: 430 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.479861:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100062 00000100:00100000:1.0:1785220581.479867:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d985e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+430:2407:x1871938519915136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.479874:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.479878:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003072b4e6/0xf62499ab92783158 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793fc expref: 430 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.479888:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003072b4e6/0xf62499ab92783158 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793fc expref: 430 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.479900:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.479903:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.479907:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.479915:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.479925:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.479941:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d985e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+430:2407:x1871938519915136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 74us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.479948:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100062 00010000:00010000:2.0:1785220581.480179:0:14164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003072b4e6/0xf62499ab92783158 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793fc expref: 429 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.480190:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510613376:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.480204:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480212:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480219:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480228:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480234:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480250:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.480412:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9573cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1028:3655:x1871938510612864:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11488us (11808us total) trans 4295069878 rc 0/0 00000100:00100000:2.0:1785220581.480420:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284130 00010000:00010000:2.0:1785220581.480445:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480455:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480463:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480471:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480485:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1785220581.480530:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b033480/2, rc 0, transno 4295069878 00000100:00100000:1.0:1785220581.480543:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510612864:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.480559:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1028:3653:x1871938510612352:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16915us (17136us total) trans 4295069879 rc 0/0 00000100:00100000:2.0:1785220581.480565:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284127 00000004:00080000:1.0:1785220581.480568:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c681180/2, rc 0, transno 4295069879 00000100:00100000:1.0:1785220581.480572:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510612352:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.480587:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.480593:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480600:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.480604:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480612:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480625:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480633:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a4814 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480649:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220581.480651:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510616832:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.480652:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480658:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.480661:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480668:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480678:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.480681:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.480686:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a47ff lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480699:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.480700:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.480702:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.480705:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.480711:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510616832 00000100:00100000:1.0:1785220581.480711:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510616704:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.480714:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510616832 00000100:00100000:1.0:1785220581.480730:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.480738:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510616704 00000100:00100000:2.0:1785220581.480740:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510616704 00000100:00100000:2.0:1785220581.480756:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284144 00000100:00100000:2.0:1785220581.480774:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1029:3655:x1871938510616832:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.480793:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29117 count 1 00010000:00010000:2.0:1785220581.480810:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.480820:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93638700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+428:2407:x1871938519914880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1440us (1994us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.480822:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000535072b8/0xf62499ab92783985 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7979f expref: 428 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.480828:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100061 00010000:00010000:2.0:1785220581.480830:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71bd:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.480833:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.480837:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480860:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000535072b8/0xf62499ab92783985 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7979f expref: 428 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480872:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000535072b8/0xf62499ab92783985 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7979f expref: 428 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.480884:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510616960:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.480916:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.480931:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284145 00000100:00100000:2.0:1785220581.480936:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1029:3655:x1871938510616704:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.480947:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29116 count 1 00010000:00010000:2.0:1785220581.480954:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480962:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4573565/0xf62499ab92783889 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79728 expref: 427 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480967:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71bc:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.480969:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.480971:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480981:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a4573565/0xf62499ab92783889 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79728 expref: 427 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.480986:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a4573565/0xf62499ab92783889 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79728 expref: 427 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.480992:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510617088:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.481004:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.481013:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.481262:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.481325:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.481330:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915008 00000100:00100000:1.0:1785220581.481335:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915008 00010000:00010000:1.0:1785220581.481354:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d8983b33/0xf62499ab92783238 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79465 expref: 428 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.481385:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915264 00000100:00100000:1.0:1785220581.481388:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915264 00010000:00010000:1.0:1785220581.481401:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b892444f/0xf62499ab92786b03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aef2 expref: 376 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.481428:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100063 00000100:00100000:1.0:1785220581.481434:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+428:2406:x1871938519915008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.481442:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.481447:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d8983b33/0xf62499ab92783238 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79465 expref: 428 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.481457:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d8983b33/0xf62499ab92783238 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79465 expref: 428 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.481473:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8983b33/0xf62499ab92783238 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79465 expref: 428 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.481483:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100064 00010000:00010000:1.0:1785220581.481483:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.481486:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.481490:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.481492:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+376:2406:x1871938519915264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.481499:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481501:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.481506:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b892444f/0xf62499ab92786b03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aef2 expref: 376 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481515:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b892444f/0xf62499ab92786b03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aef2 expref: 376 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481529:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b892444f/0xf62499ab92786b03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aef2 expref: 376 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481540:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.481543:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.481547:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481555:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481590:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.481612:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+375:2406:x1871938519915264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 122us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.481714:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100064 00010000:00010000:3.0:1785220581.481739:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481748:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481757:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481775:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481791:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.481913:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1108:3653:x1871938510613120:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12889us (13113us total) trans 4295070479 rc 0/0 00000100:00100000:3.0:1785220581.481920:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284131 00010000:00010000:3.0:1785220581.481939:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.481944:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481952:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.481956:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481964:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481975:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481981:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e680526c/0xf62499ab927a481b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.481997:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.481998:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.482000:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.482492:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.482522:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2406:x1871938519915008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1087us (1260us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.482530:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100063 00000004:00080000:1.0:1785220581.482607:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be7480/2, rc 0, transno 4295070479 00000100:00100000:1.0:1785220581.482619:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510613120:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.482680:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510617216:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.482711:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.483287:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483306:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483316:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483323:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483337:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.483471:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1030:3653:x1871938510612992:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14633us (14929us total) trans 4295069880 rc 0/0 00000100:00100000:2.0:1785220581.483479:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284129 00000004:00080000:1.0:1785220581.483548:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be4380/2, rc 0, transno 4295069880 00000100:00100000:1.0:1785220581.483559:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510612992:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.483564:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510617216 00000100:00100000:2.0:1785220581.483569:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510617216 00000100:00100000:1.0:1785220581.483597:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220581.483598:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284146 00000100:00100000:1.0:1785220581.483604:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510617344:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.483605:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1108:3652:x1871938510617216:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.483624:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29179 count 1 00000100:00100000:1.0:1785220581.483627:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.483643:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483659:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006d91d8df/0xf62499ab927872a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b241 expref: 374 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483667:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71fb:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.483672:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.483676:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483699:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006d91d8df/0xf62499ab927872a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b241 expref: 374 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483711:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006d91d8df/0xf62499ab927872a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b241 expref: 374 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.483723:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:13867:1871938510617472:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.483787:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.483802:0:14186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510617344 00000100:00100000:2.0:1785220581.483805:0:14186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510617344 00000100:00100000:2.0:1785220581.483836:0:14186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284147 00000100:00100000:2.0:1785220581.483843:0:14186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1030:3652:x1871938510617344:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.483855:0:14186:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29118 count 1 00010000:00010000:2.0:1785220581.483868:0:14186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483880:0:14186:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000f59e794/0xf62499ab92783a3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7980f expref: 426 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483887:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71be:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.483890:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.483894:0:14186:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483913:0:14186:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000f59e794/0xf62499ab92783a3b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7980f expref: 426 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.483922:0:14186:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000f59e794/0xf62499ab92783a3b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7980f expref: 426 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.483932:0:14186:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510617600:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.483966:0:14186:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896180 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.484165:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.484172:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.484180:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffa00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.484184:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.484198:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.487422:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.487433:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a480d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.487455:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.487457:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.487460:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.487615:0:14170:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510615168:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.487641:0:14170:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.487777:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.487802:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915520 00000100:00100000:1.0:1785220581.487809:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915520 00010000:00010000:1.0:1785220581.487898:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c18eb213/0xf62499ab92783311 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794b2 expref: 427 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.487931:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100065 00000100:00100000:1.0:1785220581.487940:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2404:x1871938519915520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.487954:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.487959:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c18eb213/0xf62499ab92783311 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794b2 expref: 427 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.487969:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c18eb213/0xf62499ab92783311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794b2 expref: 427 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.487986:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.487991:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.487997:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.488007:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.488025:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.488036:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510616448:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.488050:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.488070:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2404:x1871938519915520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 131us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.488072:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510616960:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.488077:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100065 00010000:00010000:2.0:1785220581.488082:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.488808:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.488849:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.488866:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.488880:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.488897:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915904 00000100:00100000:1.0:1785220581.488902:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915904 00000100:00100000:3.0:1785220581.488910:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9363aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510615552:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.488920:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000016ee4909/0xf62499ab92783510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b795bc expref: 427 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.488926:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.488940:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915392 00000100:00100000:1.0:1785220581.488943:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915392 00010000:00010000:1.0:1785220581.488953:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000443b641a/0xf62499ab92786ddb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b049 expref: 375 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.488981:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915648 00000100:00100000:1.0:1785220581.488985:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915648 00010000:00010000:1.0:1785220581.488998:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001a927c28/0xf62499ab927836bb lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7965d expref: 428 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.489015:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916160 00000100:00100000:1.0:1785220581.489018:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916160 00010000:00010000:1.0:1785220581.489030:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000535072b8/0xf62499ab92783985 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7979f expref: 429 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.489036:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100066 00000100:00100000:2.0:1785220581.489044:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+429:2404:x1871938519915904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.489050:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100067 00010000:00010000:2.0:1785220581.489056:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220581.489056:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+375:2404:x1871938519915392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.489061:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000016ee4909/0xf62499ab92783510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b795bc expref: 429 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489063:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.489067:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000443b641a/0xf62499ab92786ddb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b049 expref: 375 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489071:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000016ee4909/0xf62499ab92783510 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b795bc expref: 429 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489077:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000443b641a/0xf62499ab92786ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b049 expref: 375 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489148:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016ee4909/0xf62499ab92783510 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b795bc expref: 429 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489162:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.489163:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000443b641a/0xf62499ab92786ddb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b049 expref: 375 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489166:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.489171:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489173:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.489176:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.489179:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489181:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489187:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489199:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.489231:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+428:2404:x1871938519915904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 187us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.489237:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100066 00000100:00100000:2.0:1785220581.489253:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100068 00000100:00100000:2.0:1785220581.489259:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2404:x1871938519915648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.489267:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.489272:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001a927c28/0xf62499ab927836bb lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7965d expref: 427 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489282:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001a927c28/0xf62499ab927836bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7965d expref: 427 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489294:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.489296:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.489297:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.489301:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489309:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.489315:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+374:2404:x1871938519915392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 259us (466us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.489319:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.489321:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100067 00000100:00100000:2.0:1785220581.489333:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2404:x1871938519915648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 74us (467us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.489334:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100069 00000100:00100000:2.0:1785220581.489340:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100068 00000100:00100000:1.0:1785220581.489340:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+427:2404:x1871938519916160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.489347:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.489352:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000535072b8/0xf62499ab92783985 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7979f expref: 426 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489355:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489361:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000535072b8/0xf62499ab92783985 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7979f expref: 426 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489363:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489373:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489374:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000535072b8/0xf62499ab92783985 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7979f expref: 426 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489385:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.489387:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489388:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.489392:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489399:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489402:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.489417:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.489434:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+425:2404:x1871938519916160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (554us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.489439:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100069 00000100:00100000:2.0:1785220581.489576:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1031:3655:x1871938510614784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11242us (12929us total) trans 4295069881 rc 0/0 00000100:00100000:2.0:1785220581.489584:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284142 00010000:00010000:2.0:1785220581.489607:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489617:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489625:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489632:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.489633:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90143100/2, rc 0, transno 4295069881 00000100:00100000:1.0:1785220581.489643:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90143100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614784:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.489644:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1785220581.489710:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb1880/2, rc 0, transno 4295069882 00000100:00100000:2.0:1785220581.489711:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1031:3655:x1871938510616832:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8948us (9028us total) trans 4295069882 rc 0/0 00000100:00100000:1.0:1785220581.489716:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510616832:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.489717:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284144 00000100:00100000:1.0:1785220581.489721:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.489733:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510617728:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.489749:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510615296:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.489775:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.489776:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.489793:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510617728 00000100:00100000:2.0:1785220581.489796:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510617728 00000100:00100000:2.0:1785220581.489821:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284148 00000100:00100000:1.0:1785220581.489821:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510617856:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.489829:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1031:3655:x1871938510617728:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.489845:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:2.0:1785220581.489854:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29119 count 1 00000100:00100000:1.0:1785220581.489863:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510617856 00000100:00100000:1.0:1785220581.489866:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510617856 00010000:00010000:2.0:1785220581.489873:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489886:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2121d19/0xf62499ab92783b06 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7985c expref: 424 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.489886:0:13869:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284149 00000100:00100000:1.0:1785220581.489891:0:13869:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1032:3655:x1871938510617856:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.489894:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71bf:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.489897:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.489901:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220581.489902:0:13869:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29120 count 1 00010000:00010000:1.0:1785220581.489911:0:13869:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489921:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b2121d19/0xf62499ab92783b06 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7985c expref: 424 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489922:0:13869:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f993b69c/0xf62499ab92783bf4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798c5 expref: 424 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.489934:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b2121d19/0xf62499ab92783b06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7985c expref: 424 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489935:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c0:0x0].0x0 since=32s v=(0 0) 00010000:00010000:1.0:1785220581.489937:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.489941:0:13869:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.489946:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510617984:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.489956:0:13869:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f993b69c/0xf62499ab92783bf4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798c5 expref: 424 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.489964:0:13869:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f993b69c/0xf62499ab92783bf4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798c5 expref: 424 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.489973:0:13869:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510618112:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.489983:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.489992:0:13869:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.490001:0:14170:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.490009:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.490018:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.490026:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.490028:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.490038:0:14170:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220581.490065:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.490140:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.490156:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b033100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+1109:3653:x1871938510613888:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13635us (15077us total) trans 4295070480 rc 0/0 00000100:00100000:2.0:1785220581.490168:0:14170:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284132 00000100:00100000:2.0:1785220581.490197:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510617088:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.490209:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490350:0:14183:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c18eb213/0xf62499ab92783311 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b794b2 expref: 424 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.490403:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724e00/2, rc 0, transno 4295070480 00000100:00100000:1.0:1785220581.490415:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510613888:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.490452:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510616064:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.490468:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490476:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490482:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490489:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490495:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490508:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.490599:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfa5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1033:3653:x1871938510614528:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12895us (14072us total) trans 4295069883 rc 0/0 00000100:00100000:0.0:1785220581.490605:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284139 00010000:00010000:0.0:1785220581.490637:0:13908:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001a927c28/0xf62499ab927836bb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7965d expref: 423 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.490646:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510616320:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.490658:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490666:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490672:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490679:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490685:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.490693:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:0.0:1785220581.490758:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1033:3653:x1871938510614912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12431us (14152us total) trans 4295069884 rc 0/0 00000100:00100000:0.0:1785220581.490774:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284141 00000100:00100000:0.0:1785220581.490806:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510616576:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.490823:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.490848:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510615808:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.490861:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.491589:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916032 00000100:00100000:2.0:1785220581.491596:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916032 00000100:00100000:3.0:1785220581.491613:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916416 00000100:00100000:3.0:1785220581.491617:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916416 00010000:00010000:2.0:1785220581.491623:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006ef00579/0xf62499ab92783406 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79537 expref: 423 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.491633:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d25cb02d/0xf62499ab92787169 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b18b expref: 374 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.491654:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916672 00000100:00100000:2.0:1785220581.491657:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916672 00010000:00010000:2.0:1785220581.491670:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cdeaf63c/0xf62499ab92786cfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7afe7 expref: 375 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.491680:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100070 00000100:00100000:1.0:1785220581.491690:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d985e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+423:2406:x1871938519916032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.491695:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100071 00000100:00100000:2.0:1785220581.491699:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e11050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+375:2406:x1871938519916416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.491705:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.491706:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.491711:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d25cb02d/0xf62499ab92787169 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b18b expref: 375 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.491711:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006ef00579/0xf62499ab92783406 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79537 expref: 423 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491723:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d25cb02d/0xf62499ab92787169 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b18b expref: 375 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.491726:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006ef00579/0xf62499ab92783406 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79537 expref: 423 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491745:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d25cb02d/0xf62499ab92787169 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b18b expref: 375 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.491745:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006ef00579/0xf62499ab92783406 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79537 expref: 423 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491760:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.491760:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.491775:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.491775:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.491781:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.491781:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491792:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.491792:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491812:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.491812:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.491847:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e11050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+374:2406:x1871938519916416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 146us (1781us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.491853:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100072 00000100:00100000:2.0:1785220581.491857:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100071 00000100:00100000:3.0:1785220581.491860:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+374:2406:x1871938519916672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.491868:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.491873:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cdeaf63c/0xf62499ab92786cfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7afe7 expref: 373 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.491883:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cdeaf63c/0xf62499ab92786cfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7afe7 expref: 373 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.491893:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:13867:1871938510617472:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.491894:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cdeaf63c/0xf62499ab92786cfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7afe7 expref: 373 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.491904:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.491907:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.491909:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.491911:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.491919:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491927:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1785220581.491932:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.491935:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.491940:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fea00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.491944:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491953:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491966:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491973:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4868 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491988:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220581.491989:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.491992:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.491998:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492005:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fec00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.492008:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492009:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.492016:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492017:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.492026:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:2.0:1785220581.492027:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492032:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4876 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492042:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220581.492045:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492050:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.492053:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492060:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492068:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492074:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4822 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492114:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.492117:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492124:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.492127:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492142:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492302:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d985e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+422:2406:x1871938519916032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 613us (2272us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.492310:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100070 00000100:00100000:1.0:1785220581.492405:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93638700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510618240:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.492446:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.492495:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90140e00/2, rc 0, transno 4295069884 00010000:00010000:2.0:1785220581.492505:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492510:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90140e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614912:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.492516:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4853 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.492521:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90143b80/2, rc 0, transno 4295069883 00000100:00100000:1.0:1785220581.492524:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90143b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614528:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.492532:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.492536:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492542:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.492546:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492554:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492565:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492570:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1785220581.492572:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a4861 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492576:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510618368:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.492585:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.492588:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492595:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:1.0:1785220581.492597:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.492599:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492606:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492617:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492623:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4845 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492623:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.492630:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510618496:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.492636:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.492637:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.492639:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.492650:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510618240 00000100:00100000:1.0:1785220581.492650:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.492654:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510618240 00000100:00100000:2.0:1785220581.492676:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510618368 00000100:00100000:2.0:1785220581.492679:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510618368 00000100:00100000:2.0:1785220581.492693:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510618496 00000100:00100000:1.0:1785220581.492694:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.492695:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510618496 00000100:00100000:2.0:1785220581.492710:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284150 00000100:00100000:1.0:1785220581.492712:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916800 00000100:00100000:1.0:1785220581.492715:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916800 00000100:00100000:2.0:1785220581.492716:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1110:3652:x1871938510618240:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.492728:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a4573565/0xf62499ab92783889 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79728 expref: 422 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.492733:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29180 count 1 00000100:00100000:1.0:1785220581.492748:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100073 00010000:00010000:2.0:1785220581.492751:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492755:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d985e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+422:2404:x1871938519916800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.492774:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.492776:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2d468b0/0xf62499ab92787399 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2aa expref: 372 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.492779:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a4573565/0xf62499ab92783889 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79728 expref: 422 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492784:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71fc:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.492787:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.492789:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a4573565/0xf62499ab92783889 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79728 expref: 422 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492792:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.492803:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4573565/0xf62499ab92783889 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79728 expref: 422 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492812:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f2d468b0/0xf62499ab92787399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2aa expref: 372 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.492813:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.492816:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.492821:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.492829:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492830:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f2d468b0/0xf62499ab92787399 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2aa expref: 372 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.492845:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.492853:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510618624:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.492863:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d985e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+421:2404:x1871938519916800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (170us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.492869:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100073 00000100:00100000:2.0:1785220581.492883:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.492897:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284151 00000100:00100000:2.0:1785220581.492903:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1034:3652:x1871938510618368:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.492915:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29121 count 1 00010000:00010000:2.0:1785220581.492927:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492939:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d3fb6b7b/0xf62499ab92783cbf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79920 expref: 420 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492943:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.492946:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c1:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.492949:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.492953:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.492960:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916928 00000100:00100000:1.0:1785220581.492963:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916928 00010000:00010000:2.0:1785220581.492970:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d3fb6b7b/0xf62499ab92783cbf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79920 expref: 420 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.492976:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006d91d8df/0xf62499ab927872a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b241 expref: 373 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.492979:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d3fb6b7b/0xf62499ab92783cbf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79920 expref: 420 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.492995:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510618752:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.492995:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100074 00000100:00100000:1.0:1785220581.493001:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+373:2404:x1871938519916928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.493007:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.493012:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006d91d8df/0xf62499ab927872a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b241 expref: 373 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.493016:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.493022:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006d91d8df/0xf62499ab927872a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b241 expref: 373 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.493030:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284152 00010000:00010000:1.0:1785220581.493034:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006d91d8df/0xf62499ab927872a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b241 expref: 373 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.493037:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1034:3655:x1871938510618496:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.493044:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.493047:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fb:0x0].0x0 since=0s v=(1 0) 00002000:00080000:2.0:1785220581.493048:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29122 count 1 00010000:00010000:1.0:1785220581.493051:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493058:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493059:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493070:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004ad511cf/0xf62499ab92783df3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7999e expref: 420 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493075:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.493076:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c2:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.493079:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.493083:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493110:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+372:2404:x1871938519916928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 110us (167us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.493116:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100074 00010000:00010000:2.0:1785220581.493119:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004ad511cf/0xf62499ab92783df3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7999e expref: 420 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493128:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004ad511cf/0xf62499ab92783df3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7999e expref: 420 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.493151:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8441:1871938510618880:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.493161:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.493179:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.493191:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.493202:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493204:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519915776 00000100:00100000:1.0:1785220581.493207:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519915776 00010000:00010000:2.0:1785220581.493211:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493220:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493228:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493240:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.493300:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009b9e14c7/0xf62499ab92786ef3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b0a4 expref: 372 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493318:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916288 00000100:00100000:1.0:1785220581.493321:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916288 00010000:00010000:1.0:1785220581.493333:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000bf99e8b/0xf62499ab927871f5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1d1 expref: 373 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.493341:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1035:3655:x1871938510616704:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12404us (12610us total) trans 4295069885 rc 0/0 00000100:00100000:2.0:1785220581.493348:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284145 00000100:00100000:1.0:1785220581.493349:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100075 00000100:00100000:1.0:1785220581.493354:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e06300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+373:2406:x1871938519915776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.493362:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.493366:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009b9e14c7/0xf62499ab92786ef3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b0a4 expref: 373 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493369:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493376:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009b9e14c7/0xf62499ab92786ef3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b0a4 expref: 373 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493377:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493386:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493390:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b9e14c7/0xf62499ab92786ef3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b0a4 expref: 373 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493400:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493400:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.493402:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.493407:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493408:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220581.493414:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493431:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.493448:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e06300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+372:2406:x1871938519915776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.493454:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100075 00000100:00100000:1.0:1785220581.493467:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100076 00000100:00100000:1.0:1785220581.493473:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e05f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+371:2406:x1871938519916288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.493477:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1111:3655:x1871938510613760:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16441us (18306us total) trans 4295070482 rc 0/0 00010000:00010000:1.0:1785220581.493480:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.493483:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284133 00010000:00010000:1.0:1785220581.493485:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000bf99e8b/0xf62499ab927871f5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1d1 expref: 371 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493493:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000bf99e8b/0xf62499ab927871f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1d1 expref: 371 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493503:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493505:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.493508:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.493512:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493513:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493521:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493521:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493529:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.493533:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.493537:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.493620:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1111:3652:x1871938510617216:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10015us (10907us total) trans 4295070483 rc 0/0 00000100:00100000:2.0:1785220581.493626:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284146 00010000:00010000:2.0:1785220581.493650:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.493656:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493664:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.493667:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493675:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493687:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493687:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e05f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+371:2406:x1871938519916288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 213us (494us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.493694:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100076 00010000:00010000:2.0:1785220581.493695:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f740edce/0xf62499ab927a487d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493709:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.493712:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493719:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fee00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.493722:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.493727:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb2d80/2, rc 0, transno 4295070483 00010000:00010000:2.0:1785220581.493729:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493735:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510617216:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.493738:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493745:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4829 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493758:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.493773:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493780:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.493783:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493791:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493791:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510619008:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.493802:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.493809:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4884 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493815:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.493822:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.493824:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.493826:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.493831:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.493839:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb1500/2, rc 0, transno 4295069885 00000100:00100000:1.0:1785220581.493846:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510616704:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.493856:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ac000/2, rc 0, transno 4295070482 00000100:00100000:1.0:1785220581.493862:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510613760:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.493885:0:14186:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510617600:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.493899:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510619136:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.493901:0:14186:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.493921:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.493925:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.493943:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.493947:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510619008 00000100:00100000:1.0:1785220581.493951:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510619008 00000100:00100000:1.0:1785220581.493969:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510619136 00000100:00100000:1.0:1785220581.493972:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510619136 00000100:00100000:2.0:1785220581.493975:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.493987:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284153 00000100:00100000:1.0:1785220581.493992:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e05180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1112:3655:x1871938510619008:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.494005:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29181 count 1 00010000:00010000:1.0:1785220581.494017:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.494027:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510616192:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.494030:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f484e4df/0xf62499ab92787441 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b31a expref: 370 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494037:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71fd:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.494038:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494040:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.494044:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494062:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f484e4df/0xf62499ab92787441 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b31a expref: 370 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494070:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f484e4df/0xf62499ab92787441 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b31a expref: 370 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494106:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510619264:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.494132:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.494148:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284154 00000100:00100000:1.0:1785220581.494155:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e04700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1112:3652:x1871938510619136:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.494167:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29182 count 1 00010000:00010000:1.0:1785220581.494180:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494190:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003a989d27/0xf62499ab927874f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b36e expref: 370 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494197:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71fe:0x0].0x0 since=29s v=(0 0) 00010000:00010000:1.0:1785220581.494199:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.494203:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494297:0:14168:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf99e8b/0xf62499ab927871f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1d1 expref: 370 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.494308:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510615680:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.494321:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494329:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494336:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494337:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003a989d27/0xf62499ab927874f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b36e expref: 369 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494344:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494347:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003a989d27/0xf62499ab927874f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b36e expref: 369 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494358:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494358:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510619392:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.494372:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.494390:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.494407:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519916544 00000100:00100000:1.0:1785220581.494411:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519916544 00010000:00010000:1.0:1785220581.494428:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000084dfb94b/0xf62499ab92787027 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b129 expref: 370 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494448:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917056 00000100:00100000:1.0:1785220581.494451:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917056 00000100:00100000:2.0:1785220581.494454:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1113:3653:x1871938510614400:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16969us (18110us total) trans 4295070484 rc 0/0 00000100:00100000:2.0:1785220581.494460:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284136 00010000:00010000:1.0:1785220581.494463:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000f59e794/0xf62499ab92783a3b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7980f expref: 421 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494478:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917184 00010000:00010000:2.0:1785220581.494479:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220581.494480:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917184 00010000:00010000:2.0:1785220581.494484:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494491:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.494491:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d39859b3/0xf62499ab92786bea lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af46 expref: 371 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494495:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494503:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494506:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100077 00000100:00100000:1.0:1785220581.494511:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+371:2406:x1871938519916544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.494515:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494522:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a483e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494532:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.494535:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.494537:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.494537:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000084dfb94b/0xf62499ab92787027 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b129 expref: 371 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494538:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.494547:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000084dfb94b/0xf62499ab92787027 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b129 expref: 371 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.494549:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100078 00000100:00100000:2.0:1785220581.494556:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9adc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+421:2406:x1871938519917056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.494560:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.494563:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.494563:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.494567:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000f59e794/0xf62499ab92783a3b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7980f expref: 421 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494568:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494577:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494578:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000f59e794/0xf62499ab92783a3b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7980f expref: 421 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494589:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.494590:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f59e794/0xf62499ab92783a3b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7980f expref: 421 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494599:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.494602:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71be:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.494603:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+371:2406:x1871938519916544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (677us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.494607:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494609:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100077 00010000:00010000:2.0:1785220581.494614:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494621:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100079 00000100:00100000:1.0:1785220581.494626:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+370:2404:x1871938519917184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.494629:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.494633:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.494638:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d39859b3/0xf62499ab92786bea lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af46 expref: 370 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.494641:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9adc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+420:2406:x1871938519917056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (698us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.494648:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100078 00010000:00010000:1.0:1785220581.494649:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d39859b3/0xf62499ab92786bea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af46 expref: 370 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494659:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.494662:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.494665:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494666:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494674:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494675:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494683:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494686:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.494691:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494698:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+370:2404:x1871938519917184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 72us (723us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.494701:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.494704:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100079 00000004:00080000:1.0:1785220581.494737:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d998b9500/2, rc 0, transno 4295070484 00000100:00100000:1.0:1785220581.494746:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614400:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.494794:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1035:3652:x1871938510617344:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10951us (11165us total) trans 4295069886 rc 0/0 00000100:00100000:2.0:1785220581.494801:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284147 00000004:00080000:1.0:1785220581.494801:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb1f80/2, rc 0, transno 4295069886 00000100:00100000:1.0:1785220581.494805:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510617344:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.494809:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.494817:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510619520:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.494819:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.494823:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.494830:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffe00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:1.0:1785220581.494840:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.494865:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510619520 00000100:00100000:1.0:1785220581.494868:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510619520 00000100:00100000:1.0:1785220581.494890:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284155 00000100:00100000:1.0:1785220581.494896:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e04000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1113:3652:x1871938510619520:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.494909:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29183 count 1 00010000:00010000:1.0:1785220581.494925:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494937:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000609ea429/0xf62499ab927875ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b3bb expref: 369 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494943:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x71ff:0x0].0x0 since=29s v=(0 0) 00010000:00010000:1.0:1785220581.494946:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.494949:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494966:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000609ea429/0xf62499ab927875ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b3bb expref: 369 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.494975:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000609ea429/0xf62499ab927875ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b3bb expref: 369 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.494985:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510619648:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.495004:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df019c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.495062:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510619904:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.495237:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495246:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495263:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495270:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a488b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.495284:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+369:2406:x1871938519916672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 3423us (5143us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.495286:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.495288:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.495290:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.495293:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100072 00010000:00010000:2.0:1785220581.495296:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.495317:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000084dfb94b/0xf62499ab92787027 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b129 expref: 368 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.495326:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510615936:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.495337:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495341:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495346:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495350:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495357:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495361:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495365:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495369:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495373:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495382:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.495483:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1114:3653:x1871938510614016:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18138us (19187us total) trans 4295070485 rc 0/0 00000100:00100000:3.0:1785220581.495528:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284135 00010000:00010000:3.0:1785220581.495561:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d39859b3/0xf62499ab92786bea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af46 expref: 367 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.495573:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510615424:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.495585:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495594:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495602:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495610:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.495617:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495760:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.495861:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1114:3655:x1871938510614144:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18245us (19435us total) trans 4295070486 rc 0/0 00000100:00100000:2.0:1785220581.495867:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284138 00010000:00010000:2.0:1785220581.495885:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.495889:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495897:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.495899:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495907:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495918:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495925:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a4837 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495939:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:1.0:1785220581.495942:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.495944:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495950:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.495953:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495960:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495969:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.495970:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510619904 00000100:00100000:1.0:1785220581.495975:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510619904 00010000:00010000:2.0:1785220581.495976:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a484c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495988:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.495991:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.495997:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.496000:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.496000:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284156 00000100:00100000:1.0:1785220581.496006:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1035:3655:x1871938510619904:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.496009:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.496021:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220581.496023:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29124 count 1 00010000:00010000:2.0:1785220581.496028:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.496029:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.496031:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.496038:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.496051:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a530cd5a/0xf62499ab92783f2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a3f expref: 419 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.496059:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c4:0x0].0x0 since=32s v=(0 0) 00010000:00010000:1.0:1785220581.496062:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.496066:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.496114:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a530cd5a/0xf62499ab92783f2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a3f expref: 419 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.496125:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a530cd5a/0xf62499ab92783f2e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a3f expref: 419 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.496138:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510620032:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.496166:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.496172:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.496199:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72ed80/2, rc 0, transno 4295070485 00000100:00100000:1.0:1785220581.496281:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510614016:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.496308:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4ed80/2, rc 0, transno 4295070486 00010000:00010000:3.0:1785220581.496350:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220581.496356:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4830 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.496448:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1114:3655:x1871938510613632:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19107us (20374us total) trans 4295070487 rc 0/0 00000100:00100000:3.0:1785220581.496454:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284134 00010000:00010000:3.0:1785220581.496471:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.496476:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.497515:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1113:3655:x1871938510614272:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19945us (21123us total) trans 4295070481 rc 0/0 00000100:00100000:0.0:1785220581.497527:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284137 00010000:00010000:0.0:1785220581.497565:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.497577:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.497589:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.497603:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.497631:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.497756:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d977f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1036:3655:x1871938510614656:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19274us (20466us total) trans 4295069887 rc 0/0 00000100:00100000:0.0:1785220581.497774:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284143 00010000:00010000:2.0:1785220581.497874:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.497881:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.497889:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.497893:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.497901:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.497921:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.497929:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a486f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.497956:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.497959:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.497960:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.498154:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510619776:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.498196:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.498589:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510619776 00000100:00100000:0.0:1785220581.498596:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510619776 00000100:00100000:0.0:1785220581.498633:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284157 00000100:00100000:0.0:1785220581.498641:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1036:3654:x1871938510619776:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.498665:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29123 count 1 00010000:00010000:0.0:1785220581.498686:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.498705:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000aab21600/0xf62499ab92783e4e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799dd expref: 419 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.498713:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c3:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.498717:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.498722:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.498753:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000aab21600/0xf62499ab92783e4e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799dd expref: 419 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.498780:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000aab21600/0xf62499ab92783e4e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799dd expref: 419 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.498793:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510620416:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.498852:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.499294:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.499425:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.499446:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.499465:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917312 00000100:00100000:2.0:1785220581.499469:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917312 00010000:00010000:2.0:1785220581.499493:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001e886e8f/0xf62499ab9278378d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796b8 expref: 420 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.499517:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917568 00000100:00100000:2.0:1785220581.499520:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917568 00010000:00010000:2.0:1785220581.499534:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f993b69c/0xf62499ab92783bf4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798c5 expref: 421 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.499552:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917440 00000100:00100000:2.0:1785220581.499554:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917440 00010000:00010000:2.0:1785220581.499567:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b2121d19/0xf62499ab92783b06 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7985c expref: 422 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.499588:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100080 00000100:00100000:2.0:1785220581.499595:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+422:2405:x1871938519917312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.499605:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.499611:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001e886e8f/0xf62499ab9278378d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796b8 expref: 422 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499627:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001e886e8f/0xf62499ab9278378d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796b8 expref: 422 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499645:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001e886e8f/0xf62499ab9278378d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b796b8 expref: 422 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499660:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.499663:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.499667:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499675:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499696:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.499730:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+421:2405:x1871938519917312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.499737:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100080 00000100:00100000:2.0:1785220581.499755:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100081 00000100:00100000:2.0:1785220581.499773:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+420:2407:x1871938519917568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.499782:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.499788:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f993b69c/0xf62499ab92783bf4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798c5 expref: 420 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499804:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f993b69c/0xf62499ab92783bf4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798c5 expref: 420 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499818:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.499822:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.499826:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499834:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499847:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.499865:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+420:2407:x1871938519917568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.499871:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100081 00000100:00100000:2.0:1785220581.499885:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100082 00000100:00100000:2.0:1785220581.499892:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+419:2407:x1871938519917440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.499899:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.499904:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b2121d19/0xf62499ab92783b06 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7985c expref: 419 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499913:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b2121d19/0xf62499ab92783b06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7985c expref: 419 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499929:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.499933:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.499936:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499943:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.499954:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.499970:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+419:2407:x1871938519917440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (523us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.499976:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100082 00010000:00010000:2.0:1785220581.499997:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500007:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500017:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500025:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500040:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.500201:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfa5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1037:3653:x1871938510615040:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22412us (23643us total) trans 4295069888 rc 0/0 00000100:00100000:2.0:1785220581.500209:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284140 00010000:00010000:2.0:1785220581.500441:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.500447:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500457:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.500461:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500470:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500483:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500491:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a485a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500509:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.500511:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.500513:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.500643:0:14178:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2121d19/0xf62499ab92783b06 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7985c expref: 418 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.500656:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510617984:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.500676:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.500685:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.500692:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.500704:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.500711:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.500732:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.500872:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1037:3655:x1871938510617728:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11044us (11108us total) trans 4295069889 rc 0/0 00000100:00100000:3.0:1785220581.500879:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284148 00010000:00010000:2.0:1785220581.500929:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.500934:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500942:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.500945:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500953:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500965:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500972:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000998b8cac/0xf62499ab927a4892 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.500986:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.500987:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.500989:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.501108:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614144:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.501142:0:13869:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f993b69c/0xf62499ab92783bf4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b798c5 expref: 417 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.501154:0:13869:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510618112:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.501167:0:13869:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.501174:0:13869:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.501185:0:13869:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.501193:0:13869:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.501200:0:13869:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.501287:0:13869:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.501365:0:13869:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1037:3655:x1871938510617856:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11473us (11518us total) trans 4295069890 rc 0/0 00000100:00100000:1.0:1785220581.501372:0:13869:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284149 00000100:00100000:1.0:1785220581.501410:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510620288:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.501443:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.501481:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8d880/2, rc 0, transno 4295069888 00000100:00100000:1.0:1785220581.501492:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510615040:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.501504:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.501509:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510620160:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.501532:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.501610:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510620544:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.501635:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.501671:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4c380/2, rc 0, transno 4295070481 00010000:00010000:3.0:1785220581.501721:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.501727:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.501735:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.501739:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.501746:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.501758:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.501776:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a4899 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.501792:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.501794:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.501796:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.501855:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510620288 00000100:00100000:0.0:1785220581.501861:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510620288 00000100:00100000:0.0:1785220581.501896:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510620160 00000100:00100000:0.0:1785220581.501900:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510620160 00000100:00100000:0.0:1785220581.501918:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510620544 00000100:00100000:0.0:1785220581.501921:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510620544 00000100:00100000:0.0:1785220581.501941:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284158 00000100:00100000:0.0:1785220581.501949:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1114:3653:x1871938510620288:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.501971:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29185 count 1 00010000:00010000:0.0:1785220581.501990:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502005:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d78e76df/0xf62499ab927877ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4b0 expref: 366 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502013:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7201:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.502017:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7201:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.502021:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502052:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d78e76df/0xf62499ab927877ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4b0 expref: 366 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502064:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d78e76df/0xf62499ab927877ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4b0 expref: 366 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.502076:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510620672:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.502148:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.502164:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284159 00000100:00100000:0.0:1785220581.502172:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8ece9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1114:3653:x1871938510620160:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.502184:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29184 count 1 00010000:00010000:0.0:1785220581.502196:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502207:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fdf893ea/0xf62499ab927876e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b455 expref: 366 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502227:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7200:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.502230:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7200:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.502234:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502254:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fdf893ea/0xf62499ab927876e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b455 expref: 366 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502269:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fdf893ea/0xf62499ab927876e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b455 expref: 366 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.502278:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0000_UUID:14200:1871938510620800:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.502302:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.502318:0:14193:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284160 00000100:00100000:0.0:1785220581.502326:0:14193:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1037:3654:x1871938510620544:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.502338:0:14193:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29125 count 1 00010000:00010000:0.0:1785220581.502351:0:14193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502363:0:14193:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d14bf1db/0xf62499ab92783fdd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79aaf expref: 416 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502370:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c5:0x0].0x0 since=32s v=(0 0) 00010000:00010000:0.0:1785220581.502374:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.502377:0:14193:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502396:0:14193:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d14bf1db/0xf62499ab92783fdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79aaf expref: 416 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.502406:0:14193:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d14bf1db/0xf62499ab92783fdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79aaf expref: 416 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.502422:0:14193:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0001_UUID:14193:1871938510620928:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.502445:0:14193:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.503469:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.503599:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.503620:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.503640:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917824 00000100:00100000:2.0:1785220581.503644:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917824 00010000:00010000:2.0:1785220581.503672:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f484e4df/0xf62499ab92787441 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b31a expref: 367 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.503697:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917696 00000100:00100000:2.0:1785220581.503700:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917696 00010000:00010000:2.0:1785220581.503714:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004ad511cf/0xf62499ab92783df3 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7999e expref: 417 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.503732:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100083 00000100:00100000:2.0:1785220581.503741:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+367:2407:x1871938519917824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.503752:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.503758:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f484e4df/0xf62499ab92787441 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b31a expref: 367 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503781:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f484e4df/0xf62499ab92787441 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b31a expref: 367 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503804:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.503808:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.503814:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503824:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503841:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.503877:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+367:2407:x1871938519917824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 136us (280us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.503884:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100083 00000100:00100000:2.0:1785220581.503901:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100084 00000100:00100000:2.0:1785220581.503907:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+417:2407:x1871938519917696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.503916:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.503921:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004ad511cf/0xf62499ab92783df3 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7999e expref: 417 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503936:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004ad511cf/0xf62499ab92783df3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7999e expref: 417 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503949:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.503952:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.503956:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503976:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.503989:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.504008:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+417:2407:x1871938519917696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (387us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.504014:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100084 00000100:00100000:2.0:1785220581.504050:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.504069:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519917952 00000100:00100000:2.0:1785220581.504074:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519917952 00010000:00010000:2.0:1785220581.504113:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003a989d27/0xf62499ab927874f7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b36e expref: 367 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.504140:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100085 00000100:00100000:2.0:1785220581.504147:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+367:2407:x1871938519917952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.504155:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.504159:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003a989d27/0xf62499ab927874f7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b36e expref: 367 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.504169:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003a989d27/0xf62499ab927874f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b36e expref: 367 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.504181:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.504184:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.504188:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.504196:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.504209:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.504387:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ad511cf/0xf62499ab92783df3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7999e expref: 416 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.504402:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8441:1871938510618880:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.504421:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504431:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504439:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504449:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504457:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504474:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.504664:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1038:3655:x1871938510618496:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11627us (12012us total) trans 4295069891 rc 0/0 00000100:00100000:3.0:1785220581.504672:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284152 00010000:00010000:3.0:1785220581.504699:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.504705:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504714:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.504719:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504727:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504739:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504747:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a48ae lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.504789:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.504792:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.504793:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.505015:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+367:2407:x1871938519917952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 868us (963us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.505023:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100085 00000100:00100000:3.0:1785220581.505258:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.505279:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.505556:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510618624:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.505573:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.505602:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510618752:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.505613:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.505776:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614272:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.505807:0:14227:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f484e4df/0xf62499ab92787441 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b31a expref: 366 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.505821:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510619264:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.505836:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.505845:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.505852:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.505862:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.505871:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.505887:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.506023:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e05180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1115:3655:x1871938510619008:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12030us (12206us total) trans 4295070488 rc 0/0 00000100:00100000:1.0:1785220581.506031:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284153 00000100:00100000:1.0:1785220581.506066:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510620032:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.506079:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.506131:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510619648:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.506143:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506169:0:14267:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a989d27/0xf62499ab927874f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b36e expref: 365 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.506180:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510619392:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.506193:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506200:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506207:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506389:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506397:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506424:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.506506:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e04700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1115:3652:x1871938510619136:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12351us (12583us total) trans 4295070489 rc 0/0 00000100:00100000:1.0:1785220581.506512:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284154 00000100:00100000:1.0:1785220581.506536:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918208 00000100:00100000:1.0:1785220581.506542:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918208 00010000:00010000:1.0:1785220581.506567:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d3fb6b7b/0xf62499ab92783cbf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79920 expref: 416 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.506590:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918080 00000100:00100000:1.0:1785220581.506593:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918080 00010000:00010000:1.0:1785220581.506606:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f2d468b0/0xf62499ab92787399 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2aa expref: 365 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.506626:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100086 00000100:00100000:1.0:1785220581.506635:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+416:2405:x1871938519918208:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.506648:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.506654:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d3fb6b7b/0xf62499ab92783cbf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79920 expref: 416 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506665:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d3fb6b7b/0xf62499ab92783cbf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79920 expref: 416 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506687:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d3fb6b7b/0xf62499ab92783cbf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79920 expref: 416 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506705:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.506710:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.506716:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506725:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506747:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.506794:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+415:2405:x1871938519918208:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 160us (1536us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.506800:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100086 00000100:00100000:1.0:1785220581.506812:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100087 00000100:00100000:1.0:1785220581.506818:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+365:2405:x1871938519918080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.506826:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.506831:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f2d468b0/0xf62499ab92787399 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2aa expref: 365 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506841:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f2d468b0/0xf62499ab92787399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2aa expref: 365 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506856:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2d468b0/0xf62499ab92787399 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2aa expref: 365 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506870:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.506873:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.506877:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506884:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506900:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.506918:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2405:x1871938519918080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (1638us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.506924:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100087 00010000:00010000:1.0:1785220581.506943:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506950:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506959:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506966:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.506982:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:3.0:1785220581.507225:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.507233:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507247:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.507251:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507261:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507274:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507281:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a48b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507300:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.507303:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507310:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.507314:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507321:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507332:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507340:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a48bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507354:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.507358:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507364:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.507367:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507374:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507384:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507390:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000767f4170/0xf62499ab927a48a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.507404:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.507405:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.507407:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.507650:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1038:3652:x1871938510618368:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14746us (15050us total) trans 4295069892 rc 0/0 00000100:00100000:1.0:1785220581.507658:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284151 00010000:00010000:1.0:1785220581.507683:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.507718:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e07b80/2, rc 0, transno 4295070489 00000100:00100000:1.0:1785220581.507726:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510619136:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.507739:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb3b80/2, rc 0, transno 4295069892 00000100:00100000:1.0:1785220581.507749:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510618368:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.507832:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.507845:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510621184:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.507880:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.507925:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be4e00/2, rc 0, transno 4295070487 00000100:00100000:1.0:1785220581.507930:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510613632:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.507944:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04380/2, rc 0, transno 4295070488 00000100:00100000:1.0:1785220581.507948:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510619008:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.507956:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb0700/2, rc 0, transno 4295069891 00000100:00100000:1.0:1785220581.507962:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510618496:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.507991:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510621312:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.508038:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.508072:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb2a00/2, rc 0, transno 4295069890 00000100:00100000:1.0:1785220581.508079:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510617856:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.508330:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508341:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508350:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508359:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508375:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.508481:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1115:3652:x1871938510618240:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15764us (16034us total) trans 4295070490 rc 0/0 00000100:00100000:2.0:1785220581.508489:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284150 00000100:00100000:2.0:1785220581.508505:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510621184 00000100:00100000:2.0:1785220581.508510:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510621184 00000100:00100000:2.0:1785220581.508536:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510621312 00000100:00100000:2.0:1785220581.508539:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510621312 00000100:00100000:2.0:1785220581.508557:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284161 00000100:00100000:2.0:1785220581.508564:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e06a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1039:3653:x1871938510621184:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.508585:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29126 count 1 00010000:00010000:2.0:1785220581.508604:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508625:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d7576b80/0xf62499ab9278404d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79afc expref: 414 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508633:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c6:0x0].0x0 since=32s v=(0 0) 00010000:00010000:2.0:1785220581.508637:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.508640:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508665:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d7576b80/0xf62499ab9278404d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79afc expref: 414 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508678:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d7576b80/0xf62499ab9278404d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79afc expref: 414 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.508690:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510621568:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.508738:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.508758:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284162 00000100:00100000:2.0:1785220581.508779:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1039:3653:x1871938510621312:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.508792:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29127 count 1 00010000:00010000:2.0:1785220581.508803:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508816:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0613b16/0xf62499ab9278413b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b7a expref: 414 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508826:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c7:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.508829:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.508832:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508851:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e0613b16/0xf62499ab9278413b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b7a expref: 414 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508860:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e0613b16/0xf62499ab9278413b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b7a expref: 414 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.508869:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510621696:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.508892:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.508907:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.508912:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508920:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.508923:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508931:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508952:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508960:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a48a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.508979:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.508981:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.508983:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.508992:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.509339:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.509585:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918336 00000100:00100000:0.0:1785220581.509590:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918336 00010000:00010000:0.0:1785220581.509616:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000609ea429/0xf62499ab927875ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b3bb expref: 364 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.509642:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100088 00000100:00100000:0.0:1785220581.509650:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2407:x1871938519918336:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.509660:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.509665:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000609ea429/0xf62499ab927875ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b3bb expref: 364 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.509676:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000609ea429/0xf62499ab927875ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b3bb expref: 364 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.509692:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000609ea429/0xf62499ab927875ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b3bb expref: 364 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.509704:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.509707:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x71ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.509712:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.509726:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.509748:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.509794:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+363:2407:x1871938519918336:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 144us (455us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.509800:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100088 00000100:00100000:0.0:1785220581.509840:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510620416:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.509861:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.509977:0:14193:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0001_UUID:14193:1871938510620928:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.509990:0:14193:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.510015:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0000_UUID:14200:1871938510620800:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.510025:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.510045:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510620672:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.510056:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.510134:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510621440:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.510166:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.510195:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510621056:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.510282:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.510306:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220581.510313:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510621824:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.510334:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.510461:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510621440 00000100:00100000:3.0:1785220581.510466:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510621440 00000100:00100000:3.0:1785220581.510580:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510621056 00000100:00100000:3.0:1785220581.510584:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510621056 00000100:00100000:3.0:1785220581.510603:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510621824 00000100:00100000:3.0:1785220581.510606:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510621824 00000100:00100000:3.0:1785220581.510623:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284163 00000100:00100000:3.0:1785220581.510630:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1040:3654:x1871938510621440:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.510649:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29128 count 1 00010000:00010000:3.0:1785220581.510669:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510683:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000de0a2e05/0xf62499ab9278428b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c0d expref: 414 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510690:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c8:0x0].0x0 since=31s v=(0 0) 00010000:00010000:3.0:1785220581.510692:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.510696:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510721:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000de0a2e05/0xf62499ab9278428b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c0d expref: 414 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510732:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000de0a2e05/0xf62499ab9278428b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c0d expref: 414 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.510743:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510621952:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.510791:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.510810:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284164 00000100:00100000:3.0:1785220581.510818:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1116:3654:x1871938510621056:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.510831:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29186 count 1 00010000:00010000:3.0:1785220581.510845:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510858:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000026161d72/0xf62499ab9278782a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4fd expref: 362 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510864:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7202:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.510867:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7202:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.510871:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510891:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000026161d72/0xf62499ab9278782a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4fd expref: 362 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510900:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000026161d72/0xf62499ab9278782a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4fd expref: 362 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.510909:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0000_UUID:14162:1871938510622080:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.510934:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.510950:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284165 00000100:00100000:3.0:1785220581.510958:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1116:3654:x1871938510621824:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.510970:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29187 count 1 00010000:00010000:3.0:1785220581.510982:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.510994:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bd9e1c75/0xf62499ab92787862 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b527 expref: 362 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.511001:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7203:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.511004:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7203:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.511008:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.511027:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bd9e1c75/0xf62499ab92787862 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b527 expref: 362 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.511036:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bd9e1c75/0xf62499ab92787862 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b527 expref: 362 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.511046:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510622208:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.511068:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.511125:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.511168:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.511516:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.511540:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.511565:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.511590:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918720 00000100:00100000:0.0:1785220581.511594:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918720 00010000:00010000:0.0:1785220581.511616:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d78e76df/0xf62499ab927877ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4b0 expref: 363 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.511639:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918592 00000100:00100000:0.0:1785220581.511642:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918592 00010000:00010000:0.0:1785220581.511654:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000aab21600/0xf62499ab92783e4e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799dd expref: 415 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.511673:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918976 00000100:00100000:0.0:1785220581.511676:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918976 00010000:00010000:0.0:1785220581.511688:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fdf893ea/0xf62499ab927876e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b455 expref: 364 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.511711:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918464 00000100:00100000:0.0:1785220581.511713:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918464 00010000:00010000:0.0:1785220581.511725:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a530cd5a/0xf62499ab92783f2e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a3f expref: 416 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.511742:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519918848 00000100:00100000:0.0:1785220581.511745:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519918848 00010000:00010000:0.0:1785220581.511757:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d14bf1db/0xf62499ab92783fdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79aaf expref: 417 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.511788:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100089 00000100:00100000:0.0:1785220581.511795:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2405:x1871938519918720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.511805:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.511810:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d78e76df/0xf62499ab927877ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4b0 expref: 364 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511821:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d78e76df/0xf62499ab927877ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4b0 expref: 364 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511842:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d78e76df/0xf62499ab927877ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4b0 expref: 364 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511856:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7201:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.511860:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7201:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.511865:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511874:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511896:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.511914:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511923:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511931:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511939:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.511955:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.512123:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1117:3653:x1871938510620288:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10174us (10678us total) trans 4295070491 rc 0/0 00000100:00100000:0.0:1785220581.512131:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284158 00010000:00010000:0.0:1785220581.512155:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.512161:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.512173:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.512178:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.512186:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.512198:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.512206:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a48d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.512435:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100090 00000100:00100000:3.0:1785220581.512446:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+417:2407:x1871938519918592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.512455:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.512461:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aab21600/0xf62499ab92783e4e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799dd expref: 417 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512476:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aab21600/0xf62499ab92783e4e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799dd expref: 417 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512491:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aab21600/0xf62499ab92783e4e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b799dd expref: 417 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512502:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.512505:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.512510:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512517:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512573:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.512600:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+416:2407:x1871938519918592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 157us (1431us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.512607:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100090 00000100:00100000:3.0:1785220581.512623:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100091 00000100:00100000:3.0:1785220581.512630:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+363:2407:x1871938519918976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.512639:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.512643:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fdf893ea/0xf62499ab927876e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b455 expref: 363 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512657:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fdf893ea/0xf62499ab927876e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b455 expref: 363 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512669:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fdf893ea/0xf62499ab927876e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b455 expref: 363 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512681:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7200:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.512684:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7200:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.512688:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512695:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512712:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.512730:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+362:2407:x1871938519918976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (1214us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.512736:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100091 00000100:00100000:3.0:1785220581.512749:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100092 00000100:00100000:3.0:1785220581.512755:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+415:2407:x1871938519918464:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.512774:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.512779:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a530cd5a/0xf62499ab92783f2e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a3f expref: 415 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512793:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a530cd5a/0xf62499ab92783f2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a3f expref: 415 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512805:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a530cd5a/0xf62499ab92783f2e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79a3f expref: 415 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512815:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.512818:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.512822:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512829:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512850:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.512870:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+414:2407:x1871938519918464:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (1329us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.512875:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100092 00000100:00100000:3.0:1785220581.512888:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100093 00000100:00100000:3.0:1785220581.512896:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+413:2407:x1871938519918848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.512904:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.512908:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d14bf1db/0xf62499ab92783fdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79aaf expref: 413 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.512923:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d14bf1db/0xf62499ab92783fdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79aaf expref: 413 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513037:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d14bf1db/0xf62499ab92783fdd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79aaf expref: 413 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513048:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.513051:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.513055:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513063:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513082:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.513119:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2407:x1871938519918848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 224us (1553us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.513126:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100093 00010000:00010000:3.0:1785220581.513153:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513162:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513172:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513181:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.513196:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.513362:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513372:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513381:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513388:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513404:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.513534:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1041:3655:x1871938510619904:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17528us (18449us total) trans 4295069893 rc 0/0 00000100:00100000:1.0:1785220581.513542:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284156 00010000:00010000:1.0:1785220581.513569:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513578:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513586:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513592:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513605:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220581.513685:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e04000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1117:3652:x1871938510619520:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18789us (18843us total) trans 4295070492 rc 0/0 00000100:00100000:1.0:1785220581.513695:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284155 00010000:00010000:1.0:1785220581.513730:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.513735:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513744:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.513747:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513756:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513779:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513787:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a48df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513807:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.513811:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513817:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.513820:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513832:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513843:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513850:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a48ca lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513863:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.513867:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513873:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.513876:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513884:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513894:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513901:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a48c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.513920:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.513922:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.513924:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.513965:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbc700/2, rc 0, transno 4295070491 00000100:00100000:1.0:1785220581.513972:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510620288:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.513998:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e07480/2, rc 0, transno 4295070492 00000100:00100000:1.0:1785220581.514002:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510619520:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.514012:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93638700/2, rc 0, transno 4295070490 00000100:00100000:1.0:1785220581.514015:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93638700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510618240:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.514053:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb3480/2, rc 0, transno 4295069889 00000100:00100000:1.0:1785220581.514062:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510617728:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.514073:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90142680/2, rc 0, transno 4295069887 00000100:00100000:1.0:1785220581.514076:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90142680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510614656:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.514106:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e05c00/2, rc 0, transno 4295069893 00000100:00100000:1.0:1785220581.514111:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510619904:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.514167:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510622336:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.514202:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.514329:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.514331:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.514333:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.514345:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.514355:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.514363:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.514376:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.514389:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.514485:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1041:3654:x1871938510619776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15845us (16288us total) trans 4295069894 rc 0/0 00000100:00100000:0.0:1785220581.514493:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284157 00010000:00010000:1.0:1785220581.514549:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.514554:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.514562:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.514566:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.514575:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.514588:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.514595:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a48d1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.514611:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.514612:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.514614:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.514675:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99066a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510622464:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.514702:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.515038:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8ece9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1117:3653:x1871938510620160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12866us (13503us total) trans 4295070493 rc 0/0 00000100:00100000:3.0:1785220581.515046:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284159 00000100:00100000:0.0:1785220581.515102:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510622336 00000100:00100000:0.0:1785220581.515109:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510622336 00000100:00100000:0.0:1785220581.515145:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284166 00000100:00100000:0.0:1785220581.515153:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1041:3652:x1871938510622336:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.515176:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29129 count 1 00010000:00010000:0.0:1785220581.515195:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515226:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000541f9d9c/0xf62499ab9278433a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c7d expref: 411 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515235:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71c9:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.515239:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.515243:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515270:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000541f9d9c/0xf62499ab9278433a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c7d expref: 411 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515287:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000541f9d9c/0xf62499ab9278433a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c7d expref: 411 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.515299:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510622592:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.515345:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.515392:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+361:2405:x1871938519918720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 3597us (4266us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.515401:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100089 00000100:00100000:0.0:1785220581.515555:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.515576:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519919104 00000100:00100000:0.0:1785220581.515580:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519919104 00010000:00010000:0.0:1785220581.515601:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d7576b80/0xf62499ab9278404d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79afc expref: 412 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.515626:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100094 00000100:00100000:0.0:1785220581.515633:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2406:x1871938519919104:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.515643:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.515648:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d7576b80/0xf62499ab9278404d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79afc expref: 412 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515661:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d7576b80/0xf62499ab9278404d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79afc expref: 412 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515687:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.515690:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.515696:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515705:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515722:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.515741:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2406:x1871938519919104:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.515748:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100094 00000100:00100000:0.0:1785220581.515814:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.515832:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519919232 00000100:00100000:0.0:1785220581.515836:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519919232 00010000:00010000:0.0:1785220581.515851:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e0613b16/0xf62499ab9278413b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b7a expref: 412 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.515878:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100095 00000100:00100000:0.0:1785220581.515885:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2406:x1871938519919232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.515893:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.515898:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e0613b16/0xf62499ab9278413b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b7a expref: 412 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515908:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e0613b16/0xf62499ab9278413b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b7a expref: 412 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515921:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.515924:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.515928:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515936:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.515949:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.515967:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2406:x1871938519919232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (153us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.515974:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100095 00000004:00080000:0.0:1785220581.516146:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04e00/2, rc 0, transno 4295069894 00000100:00100000:0.0:1785220581.516152:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510619776:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.516497:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbf480/2, rc 0, transno 4295070493 00000100:00100000:1.0:1785220581.516503:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510620160:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.516576:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510622848:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.516607:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.516627:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.516632:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510622720:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.516650:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.516714:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510622464 00000100:00100000:0.0:1785220581.516719:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510622464 00000100:00100000:0.0:1785220581.516753:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510622848 00000100:00100000:0.0:1785220581.516756:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510622848 00000100:00100000:0.0:1785220581.516784:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510622720 00000100:00100000:0.0:1785220581.516787:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510622720 00000100:00100000:0.0:1785220581.516804:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284167 00000100:00100000:0.0:1785220581.516810:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1044:3653:x1871938510622464:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.516828:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29130 count 1 00010000:00010000:0.0:1785220581.516849:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.516870:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000086c5903c/0xf62499ab9278446e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d10 expref: 411 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.516877:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ca:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.516880:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.516884:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.516907:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000086c5903c/0xf62499ab9278446e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d10 expref: 411 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.516918:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000086c5903c/0xf62499ab9278446e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d10 expref: 411 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.516929:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510622976:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.516972:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.517248:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.517395:0:14261:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284168 00000100:00100000:1.0:1785220581.517405:0:14261:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1044:3655:x1871938510622848:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.517419:0:14261:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29132 count 1 00010000:00010000:1.0:1785220581.517434:0:14261:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517447:0:14261:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8cd9e4a/0xf62499ab9278467b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e05 expref: 411 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517454:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71cc:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.517457:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.517461:0:14261:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517481:0:14261:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8cd9e4a/0xf62499ab9278467b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e05 expref: 411 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517490:0:14261:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8cd9e4a/0xf62499ab9278467b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e05 expref: 411 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.517501:0:14261:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99067800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510623104:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.517538:0:14261:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.517555:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284169 00000100:00100000:1.0:1785220581.517563:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1044:3655:x1871938510622720:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.517576:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29131 count 1 00010000:00010000:1.0:1785220581.517588:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517600:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008ad3039b/0xf62499ab927845be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79da3 expref: 411 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517608:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71cb:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.517611:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.517614:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517633:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008ad3039b/0xf62499ab927845be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79da3 expref: 411 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517642:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008ad3039b/0xf62499ab927845be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79da3 expref: 411 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.517651:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99066d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510623232:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.517673:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.517688:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519919360 00000100:00100000:1.0:1785220581.517692:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519919360 00010000:00010000:1.0:1785220581.517711:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000de0a2e05/0xf62499ab9278428b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c0d expref: 412 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.517733:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100096 00000100:00100000:1.0:1785220581.517740:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2406:x1871938519919360:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.517748:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.517753:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000de0a2e05/0xf62499ab9278428b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c0d expref: 412 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517788:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000de0a2e05/0xf62499ab9278428b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c0d expref: 412 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517809:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.517812:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.517817:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517826:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.517841:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.517863:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2406:x1871938519919360:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 123us (615us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.517870:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100096 00010000:00010000:0.0:1785220581.518281:0:14172:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0613b16/0xf62499ab9278413b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79b7a expref: 411 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.518298:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510621696:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.518317:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518330:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518337:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518346:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518353:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518381:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.518533:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1045:3653:x1871938510621312:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9755us (10494us total) trans 4295069895 rc 0/0 00000100:00100000:0.0:1785220581.518540:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284162 00010000:00010000:0.0:1785220581.518578:0:14209:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7576b80/0xf62499ab9278404d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79afc expref: 410 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.518589:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510621568:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.518601:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518610:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518617:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518625:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518633:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518646:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.518717:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e06a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1045:3653:x1871938510621184:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10153us (10836us total) trans 4295069896 rc 0/0 00000100:00100000:0.0:1785220581.518724:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284161 00010000:00010000:0.0:1785220581.518779:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.518785:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518794:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.518798:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518807:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518822:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518830:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a48f4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518852:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.518856:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518862:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffa00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.518865:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518871:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518881:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518887:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b81a2b6/0xf62499ab927a48ed lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.518900:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.518902:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.518904:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.518913:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.518957:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.518962:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.518988:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e05880/2, rc 0, transno 4295069895 00000100:00100000:1.0:1785220581.519006:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510621312:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.519017:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e06680/2, rc 0, transno 4295069896 00000100:00100000:1.0:1785220581.519021:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510621184:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.519208:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510623360:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.519256:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.519662:0:14159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510623360 00000100:00100000:2.0:1785220581.519669:0:14159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510623360 00000100:00100000:2.0:1785220581.519710:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284170 00000100:00100000:2.0:1785220581.519718:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1045:3652:x1871938510623360:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.519740:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29133 count 1 00010000:00010000:2.0:1785220581.519758:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.519786:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000523f3ec7/0xf62499ab92784803 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e8a expref: 409 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.519794:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71cd:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.519798:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.519801:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.519833:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000523f3ec7/0xf62499ab92784803 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e8a expref: 409 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.519846:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000523f3ec7/0xf62499ab92784803 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e8a expref: 409 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.519858:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510623488:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.519910:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.520029:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.520188:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510622592:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.520210:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.520266:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510623744:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.520296:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.520657:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.520665:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510623616:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.520697:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.520739:0:14204:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510623744 00000100:00100000:1.0:1785220581.520744:0:14204:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510623744 00000100:00100000:1.0:1785220581.520784:0:14204:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510623616 00000100:00100000:1.0:1785220581.520788:0:14204:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510623616 00000100:00100000:1.0:1785220581.520809:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284171 00000100:00100000:1.0:1785220581.520816:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1046:3654:x1871938510623744:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.520833:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29134 count 1 00010000:00010000:1.0:1785220581.520851:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.520865:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005fb79335/0xf62499ab927848f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ef3 expref: 409 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.520872:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ce:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.520877:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.520881:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.520906:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005fb79335/0xf62499ab927848f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ef3 expref: 409 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.520916:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005fb79335/0xf62499ab927848f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ef3 expref: 409 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.520927:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510623872:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.520960:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01300 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.520978:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284172 00000100:00100000:1.0:1785220581.520985:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1117:3652:x1871938510623616:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.521020:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29188 count 1 00010000:00010000:1.0:1785220581.521035:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.521048:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000429cb9b5/0xf62499ab92787950 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5a5 expref: 360 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.521055:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7204:0x0].0x0 since=29s v=(0 0) 00010000:00010000:1.0:1785220581.521058:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7204:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.521063:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.521082:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000429cb9b5/0xf62499ab92787950 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5a5 expref: 360 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.521115:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000429cb9b5/0xf62499ab92787950 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5a5 expref: 360 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.521126:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510624000:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.521150:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01b40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.521169:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519919488 00000100:00100000:1.0:1785220581.521173:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519919488 00010000:00010000:1.0:1785220581.521192:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000026161d72/0xf62499ab9278782a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4fd expref: 361 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.521379:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510624256:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.521408:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.521457:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510624384:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.521491:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.521542:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510624512:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.521563:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.521583:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510624256 00000100:00100000:2.0:1785220581.521587:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510624256 00000100:00100000:2.0:1785220581.521607:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510624384 00000100:00100000:2.0:1785220581.521611:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510624384 00000100:00100000:2.0:1785220581.521623:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510624512 00000100:00100000:2.0:1785220581.521626:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510624512 00000100:00100000:2.0:1785220581.521642:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284173 00000100:00100000:2.0:1785220581.521648:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1120:3652:x1871938510624256:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.521662:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29190 count 1 00010000:00010000:2.0:1785220581.521675:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521688:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e7e3453a/0xf62499ab92787b09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b69a expref: 361 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521694:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7206:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.521697:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7206:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.521701:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521721:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e7e3453a/0xf62499ab92787b09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b69a expref: 361 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521730:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e7e3453a/0xf62499ab92787b09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b69a expref: 361 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.521741:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510624768:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.521786:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896180 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.521817:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284174 00000100:00100000:2.0:1785220581.521824:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1120:3653:x1871938510624384:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.521835:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29191 count 1 00010000:00010000:2.0:1785220581.521846:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521858:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e1f278f1/0xf62499ab92787cb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6fc expref: 361 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521870:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7207:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.521874:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7207:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.521877:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521894:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e1f278f1/0xf62499ab92787cb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6fc expref: 361 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521903:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e1f278f1/0xf62499ab92787cb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6fc expref: 361 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.521912:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510624896:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.521933:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.521949:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284175 00000100:00100000:2.0:1785220581.521956:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1120:3654:x1871938510624512:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.521967:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29192 count 1 00010000:00010000:2.0:1785220581.521980:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521991:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006c3d9445/0xf62499ab92787da9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b796 expref: 361 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.521997:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7208:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.521999:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7208:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.522003:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.522020:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006c3d9445/0xf62499ab92787da9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b796 expref: 361 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.522029:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006c3d9445/0xf62499ab92787da9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b796 expref: 361 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.522039:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510625024:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.522061:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.522080:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1785220581.522109:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510624128:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.522133:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.522156:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510624128 00000100:00100000:2.0:1785220581.522160:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510624128 00000100:00100000:2.0:1785220581.522186:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284176 00000100:00100000:2.0:1785220581.522192:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1121:3654:x1871938510624128:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.522204:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29189 count 1 00000100:00100000:1.0:1785220581.522320:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100097 00000100:00100000:1.0:1785220581.522329:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+361:2406:x1871938519919488:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.522338:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.522345:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000026161d72/0xf62499ab9278782a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4fd expref: 361 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.522357:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000026161d72/0xf62499ab9278782a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4fd expref: 361 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.522376:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7202:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.522379:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7202:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.522385:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.522393:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.522408:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.522430:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+361:2406:x1871938519919488:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (2401us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.522436:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100097 00010000:00010000:2.0:1785220581.522724:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.522737:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e79bb9e7/0xf62499ab92787a0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b61c expref: 360 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.522745:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7205:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.522748:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7205:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.522752:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.522783:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e79bb9e7/0xf62499ab92787a0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b61c expref: 360 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.522793:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e79bb9e7/0xf62499ab92787a0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b61c expref: 360 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.522804:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ad880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510625152:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.522917:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.522936:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510624640:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.522962:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.523030:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510624640 00000100:00100000:1.0:1785220581.523035:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510624640 00000100:00100000:1.0:1785220581.523070:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284177 00000100:00100000:1.0:1785220581.523077:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1122:3654:x1871938510624640:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.523113:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29193 count 1 00010000:00010000:1.0:1785220581.523127:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.523140:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eb78eedc/0xf62499ab92787e97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b814 expref: 360 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.523147:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7209:0x0].0x0 since=28s v=(0 0) 00010000:00010000:1.0:1785220581.523150:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7209:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.523153:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.523172:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eb78eedc/0xf62499ab92787e97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b814 expref: 360 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.523182:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eb78eedc/0xf62499ab92787e97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b814 expref: 360 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.523192:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510625280:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.523291:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df019c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.523732:0:14193:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.523743:0:14193:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.523752:0:14193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.523771:0:14193:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.523797:0:14193:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.523958:0:14193:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1047:3654:x1871938510620544:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21633us (22321us total) trans 4295069897 rc 0/0 00000100:00100000:3.0:1785220581.523966:0:14193:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284160 00010000:00010000:2.0:1785220581.524103:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.524111:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.524120:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.524124:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.524134:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.524153:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.524162:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a48e6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.524182:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.524184:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.524186:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.524286:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e07800/2, rc 0, transno 4295069897 00000100:00100000:2.0:1785220581.524301:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510620544:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.524363:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510625408:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.524396:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.524497:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510625408 00000100:00100000:1.0:1785220581.524502:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510625408 00000100:00100000:1.0:1785220581.524543:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284178 00000100:00100000:1.0:1785220581.524551:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1047:3652:x1871938510625408:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.524567:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29135 count 1 00010000:00010000:1.0:1785220581.524583:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.524597:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e12d24d0/0xf62499ab927849f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f63 expref: 409 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.524605:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71cf:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.524609:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.524614:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.524636:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e12d24d0/0xf62499ab927849f4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f63 expref: 409 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.524648:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e12d24d0/0xf62499ab927849f4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f63 expref: 409 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.524661:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510625536:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.524699:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.524914:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510622976:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.524939:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.524976:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.525012:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.525037:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.525112:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.525136:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.525154:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519919744 00000100:00100000:0.0:1785220581.525158:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519919744 00010000:00010000:0.0:1785220581.525179:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000541f9d9c/0xf62499ab9278433a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c7d expref: 410 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.525207:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519919616 00000100:00100000:2.0:1785220581.525330:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510623488:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.525345:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.525365:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519919872 00000100:00100000:2.0:1785220581.525368:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519919872 00010000:00010000:2.0:1785220581.525384:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000086c5903c/0xf62499ab9278446e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d10 expref: 411 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.525406:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920000 00000100:00100000:2.0:1785220581.525409:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920000 00010000:00010000:2.0:1785220581.525422:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f8cd9e4a/0xf62499ab9278467b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e05 expref: 412 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.525438:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920256 00000100:00100000:2.0:1785220581.525441:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920256 00010000:00010000:2.0:1785220581.525453:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000523f3ec7/0xf62499ab92784803 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e8a expref: 413 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.525474:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100098 00000100:00100000:2.0:1785220581.525482:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+413:2407:x1871938519919744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.525492:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.525497:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000541f9d9c/0xf62499ab9278433a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c7d expref: 413 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525509:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000541f9d9c/0xf62499ab9278433a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c7d expref: 413 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525527:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000541f9d9c/0xf62499ab9278433a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c7d expref: 413 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525540:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525544:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525549:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525558:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525577:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.525603:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+412:2407:x1871938519919744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 122us (627us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.525610:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100098 00000100:00100000:2.0:1785220581.525626:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100099 00000100:00100000:2.0:1785220581.525632:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+411:2407:x1871938519919872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.525639:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.525644:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000086c5903c/0xf62499ab9278446e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d10 expref: 411 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525653:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000086c5903c/0xf62499ab9278446e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d10 expref: 411 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525667:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000086c5903c/0xf62499ab9278446e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79d10 expref: 411 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525677:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525680:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525683:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525690:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525706:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.525723:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+410:2407:x1871938519919872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (685us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.525729:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100099 00000100:00100000:2.0:1785220581.525745:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100100 00000100:00100000:2.0:1785220581.525752:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+409:2407:x1871938519920000:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.525760:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.525778:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f8cd9e4a/0xf62499ab9278467b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e05 expref: 409 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525789:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f8cd9e4a/0xf62499ab9278467b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e05 expref: 409 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525801:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525804:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525808:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525815:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525827:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.525843:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+409:2407:x1871938519920000:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (731us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.525849:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100100 00000100:00100000:2.0:1785220581.525863:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100101 00000100:00100000:2.0:1785220581.525869:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+408:2407:x1871938519920256:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.525876:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.525881:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000523f3ec7/0xf62499ab92784803 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e8a expref: 408 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525892:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000523f3ec7/0xf62499ab92784803 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e8a expref: 408 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525904:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000523f3ec7/0xf62499ab92784803 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e8a expref: 408 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525914:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525917:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.525920:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525927:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.525942:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.525959:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+407:2407:x1871938519920256:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (823us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.525967:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100101 00010000:00010000:2.0:1785220581.525992:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.526000:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.526010:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.526018:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.526034:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.526301:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1048:3652:x1871938510623360:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6583us (7044us total) trans 4295069898 rc 0/0 00000100:00100000:2.0:1785220581.526309:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284170 00000004:00080000:2.0:1785220581.526349:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbd180/2, rc 0, transno 4295069898 00000100:00100000:2.0:1785220581.526360:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510623360:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.526423:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510625664:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.526453:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.526537:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519919616 00010000:00010000:0.0:1785220581.526563:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bd9e1c75/0xf62499ab92787862 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b527 expref: 361 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.526590:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100102 00000100:00100000:0.0:1785220581.526598:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9af850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+361:2407:x1871938519919616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.526607:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.526611:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bd9e1c75/0xf62499ab92787862 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b527 expref: 361 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526620:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bd9e1c75/0xf62499ab92787862 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b527 expref: 361 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526634:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7203:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.526637:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7203:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.526640:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526648:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526662:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.526683:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9af850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+361:2407:x1871938519919616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (1670us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.526690:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100102 00010000:00010000:0.0:1785220581.526733:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526742:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526756:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526775:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526797:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.526888:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1048:3652:x1871938510622336:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11736us (12685us total) trans 4295069899 rc 0/0 00000100:00100000:0.0:1785220581.526895:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284166 00000100:00100000:0.0:1785220581.526908:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510625664 00000100:00100000:0.0:1785220581.526912:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510625664 00000100:00100000:0.0:1785220581.526937:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284179 00000100:00100000:0.0:1785220581.526943:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1048:3653:x1871938510625664:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.526960:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29136 count 1 00010000:00010000:0.0:1785220581.526981:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.526995:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000037091e1e/0xf62499ab92784afe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79fe1 expref: 406 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527002:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d0:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.527005:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.527008:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527036:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000037091e1e/0xf62499ab92784afe lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79fe1 expref: 406 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527048:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000037091e1e/0xf62499ab92784afe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79fe1 expref: 406 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.527060:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510625792:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.527105:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.527133:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527143:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527150:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527157:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527173:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220581.527318:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.527323:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527329:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffe00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.527332:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527339:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527347:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527355:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000663e34e8/0xf62499ab927a492c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527369:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.527371:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527375:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.527377:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527381:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527386:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527390:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4910 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527398:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.527400:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527403:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.527405:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527410:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527414:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527418:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4917 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527426:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.527427:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.527428:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.527446:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e05f80/2, rc 0, transno 4295069899 00000100:00100000:2.0:1785220581.527452:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510622336:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.527485:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510625920:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.527503:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.527520:0:14164:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510625920 00000100:00100000:2.0:1785220581.527522:0:14164:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510625920 00000100:00100000:2.0:1785220581.527539:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284180 00000100:00100000:2.0:1785220581.527543:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1049:3654:x1871938510625920:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.527551:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29137 count 1 00010000:00010000:2.0:1785220581.527560:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527572:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2a09df3/0xf62499ab92784c24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a066 expref: 406 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527576:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d1:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.527578:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.527580:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527591:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c2a09df3/0xf62499ab92784c24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a066 expref: 406 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.527596:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c2a09df3/0xf62499ab92784c24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a066 expref: 406 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.527602:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510626048:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.527619:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.527716:0:14261:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8cd9e4a/0xf62499ab9278467b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e05 expref: 406 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.527725:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99067800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510623104:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.527738:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527747:0:14261:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527751:0:14261:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527756:0:14261:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527760:0:14261:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527781:0:14261:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.527859:0:14261:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1051:3655:x1871938510622848:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10456us (11252us total) trans 4295069901 rc 0/0 00000100:00100000:1.0:1785220581.527866:0:14261:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284168 00000100:00100000:1.0:1785220581.527898:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99066d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510623232:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.527909:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527928:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1785220581.527928:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1050:3653:x1871938510622464:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11118us (13225us total) trans 4295069900 rc 0/0 00010000:00010000:1.0:1785220581.527932:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.527935:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284167 00010000:00010000:1.0:1785220581.527944:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.527947:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527955:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.527955:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.527966:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527973:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a491e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.527987:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.527988:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.527990:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.528021:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99065500/2, rc 0, transno 4295069901 00000100:00100000:1.0:1785220581.528032:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510622848:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.528377:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.528398:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99066a00/2, rc 0, transno 4295069900 00000100:00100000:2.0:1785220581.528407:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99066a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510622464:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.528419:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.528427:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99066d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510626176:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.528453:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.528478:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510626176 00000100:00100000:2.0:1785220581.528482:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510626176 00000100:00100000:2.0:1785220581.528510:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284181 00000100:00100000:2.0:1785220581.528517:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1050:3654:x1871938510626176:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.528531:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29138 count 1 00010000:00010000:2.0:1785220581.528544:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.528556:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000046e646aa/0xf62499ab92784cda lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0ba expref: 405 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.528563:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d2:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.528566:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.528569:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.528587:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000046e646aa/0xf62499ab92784cda lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0ba expref: 405 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.528597:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000046e646aa/0xf62499ab92784cda lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0ba expref: 405 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.528608:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510626304:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.528638:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.528805:0:14256:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000de0a2e05/0xf62499ab9278428b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79c0d expref: 405 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.528825:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510621952:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.528848:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.528857:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.528864:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.528873:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.528882:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.528903:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.528975:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99067800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510626432:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.529001:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.529017:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1051:3654:x1871938510621440:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18388us (18850us total) trans 4295069902 rc 0/0 00000100:00100000:3.0:1785220581.529025:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284163 00000100:00100000:3.0:1785220581.529041:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510626432 00000100:00100000:3.0:1785220581.529045:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510626432 00000100:00100000:3.0:1785220581.529075:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284182 00000100:00100000:3.0:1785220581.529082:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1051:3652:x1871938510626432:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.529249:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29139 count 1 00010000:00010000:3.0:1785220581.529262:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529274:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa95305a/0xf62499ab92784d89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a12a expref: 404 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529281:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d3:0x0].0x0 since=31s v=(0 0) 00010000:00010000:3.0:1785220581.529284:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.529287:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529308:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fa95305a/0xf62499ab92784d89 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a12a expref: 404 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529318:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fa95305a/0xf62499ab92784d89 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a12a expref: 404 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.529328:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510626560:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.529406:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.529412:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529426:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.529429:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529437:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529452:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529459:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a48fb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529477:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.529479:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.529481:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.529513:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e06300/2, rc 0, transno 4295069902 00000100:00100000:2.0:1785220581.529522:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510621440:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.529601:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.529610:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99066680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510626688:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.529635:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.529684:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.529684:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510626688 00000100:00100000:2.0:1785220581.529689:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510626688 00000100:00100000:2.0:1785220581.529713:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284183 00000100:00100000:2.0:1785220581.529724:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1052:3652:x1871938510626688:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.529737:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29140 count 1 00010000:00010000:2.0:1785220581.529748:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529776:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000df6dcb55/0xf62499ab92784e62 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a17e expref: 404 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529783:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d4:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.529786:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.529790:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529809:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000df6dcb55/0xf62499ab92784e62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a17e expref: 404 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.529818:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000df6dcb55/0xf62499ab92784e62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a17e expref: 404 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.529828:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510626816:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.529861:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.529899:0:14162:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000026161d72/0xf62499ab9278782a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4fd expref: 360 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.529910:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0000_UUID:14162:1871938510622080:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.529926:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529935:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529942:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529950:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529957:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.529976:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.529990:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.529996:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.530004:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.530007:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.530014:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.530025:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.530031:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4902 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.530046:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.530047:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.530049:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.530080:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.530124:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1123:3654:x1871938510621056:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19306us (19841us total) trans 4295070494 rc 0/0 00000100:00100000:3.0:1785220581.530131:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284164 00000004:00080000:2.0:1785220581.530155:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbd180/2, rc 0, transno 4295070494 00010000:00010000:3.0:1785220581.530161:0:14250:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd9e1c75/0xf62499ab92787862 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b527 expref: 359 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.530165:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510621056:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.530172:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510622208:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.530181:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.530190:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.530197:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.530200:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510626944:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.530206:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.530213:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.530230:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.530298:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.530300:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1123:3654:x1871938510621824:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19343us (19964us total) trans 4295070495 rc 0/0 00000100:00100000:3.0:1785220581.530306:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284165 00000100:00100000:3.0:1785220581.530318:0:14250:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510626944 00000100:00100000:3.0:1785220581.530321:0:14250:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510626944 00000004:00080000:2.0:1785220581.530333:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72d500/2, rc 0, transno 4295070495 00000100:00100000:2.0:1785220581.530344:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510621824:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.530346:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284184 00000100:00100000:3.0:1785220581.530352:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1123:3654:x1871938510626944:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.530364:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29194 count 1 00000100:00100000:2.0:1785220581.530372:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:3.0:1785220581.530376:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.530378:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510627072:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.530387:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000053d0b2b9/0xf62499ab92787f3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b884 expref: 358 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.530394:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x720a:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.530398:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720a:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220581.530399:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.530401:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.530417:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510627072 00010000:00010000:3.0:1785220581.530420:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000053d0b2b9/0xf62499ab92787f3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b884 expref: 358 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.530420:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510627072 00010000:00010000:3.0:1785220581.530430:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000053d0b2b9/0xf62499ab92787f3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b884 expref: 358 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.530441:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510627200:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.530443:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284185 00000100:00100000:2.0:1785220581.530448:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1124:3654:x1871938510627072:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.530461:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29195 count 1 00000100:00100000:3.0:1785220581.530467:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.530471:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.530482:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007dd522b6/0xf62499ab92787fc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8df expref: 358 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.530488:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x720b:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.530491:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.530494:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.530512:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007dd522b6/0xf62499ab92787fc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8df expref: 358 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.530521:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007dd522b6/0xf62499ab92787fc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8df expref: 358 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.530531:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510627328:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.530552:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.530969:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.530987:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920128 00000100:00100000:0.0:1785220581.530990:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920128 00010000:00010000:0.0:1785220581.531012:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008ad3039b/0xf62499ab927845be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79da3 expref: 405 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.531039:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920384 00000100:00100000:0.0:1785220581.531042:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920384 00010000:00010000:0.0:1785220581.531055:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005fb79335/0xf62499ab927848f1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ef3 expref: 406 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.531073:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100103 00000100:00100000:0.0:1785220581.531079:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+406:2407:x1871938519920128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.531110:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.531117:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008ad3039b/0xf62499ab927845be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79da3 expref: 406 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.531126:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008ad3039b/0xf62499ab927845be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79da3 expref: 406 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.531205:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008ad3039b/0xf62499ab927845be lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79da3 expref: 406 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.531428:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.531432:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.531438:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.531448:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.531479:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.531560:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100104 00000100:00100000:2.0:1785220581.531569:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+405:2406:x1871938519920384:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.531578:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.531583:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005fb79335/0xf62499ab927848f1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ef3 expref: 405 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.531593:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005fb79335/0xf62499ab927848f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ef3 expref: 405 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.531606:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.531609:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.531612:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.531620:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.531634:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.532637:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+405:2407:x1871938519920128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1554us (2553us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.532647:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100103 00000100:00100000:2.0:1785220581.532726:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+404:2406:x1871938519920384:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1158us (1758us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.532733:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100104 00000100:00100000:3.0:1785220581.532817:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.532845:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920640 00000100:00100000:3.0:1785220581.532850:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920640 00000100:00100000:2.0:1785220581.533018:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510624896:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.533040:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.533071:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510625024:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.533082:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.533145:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510624768:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.533156:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533328:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.533336:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533345:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.533350:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533359:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533375:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533382:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4909 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533405:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.533407:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.533408:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.533434:0:14204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fb79335/0xf62499ab927848f1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79ef3 expref: 403 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.533446:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510623872:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.533460:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533468:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533475:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533484:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533491:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533515:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.533651:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006c3d9445/0xf62499ab92787da9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b796 expref: 359 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.533668:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93665500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1053:3654:x1871938510623744:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12852us (13370us total) trans 4295069903 rc 0/0 00000100:00100000:1.0:1785220581.533676:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284171 00000100:00100000:3.0:1785220581.533677:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100105 00000100:00100000:3.0:1785220581.533684:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+359:2406:x1871938519920640:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.533696:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.533701:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006c3d9445/0xf62499ab92787da9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b796 expref: 359 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.533705:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510624000:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.533711:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006c3d9445/0xf62499ab92787da9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b796 expref: 359 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533717:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533733:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.533734:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c3d9445/0xf62499ab92787da9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b796 expref: 359 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533743:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.533746:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7208:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.533750:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7208:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.533750:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.533755:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533758:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.533776:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533785:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.533794:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.533819:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+358:2406:x1871938519920640:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 136us (1003us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.533826:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100105 00000100:00100000:1.0:1785220581.533871:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1053:3655:x1871938510622720:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16309us (17220us total) trans 4295069904 rc 0/0 00000100:00100000:1.0:1785220581.533878:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284169 00010000:00010000:1.0:1785220581.533894:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.533899:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533907:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.533910:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533918:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533930:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533937:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4933 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533951:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.533955:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533960:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.533963:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533970:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533980:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533987:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4925 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.533999:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.534001:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.534002:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.534036:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.534062:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.534076:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.534183:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510625280:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.534195:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.534476:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920896 00000100:00100000:2.0:1785220581.534482:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920896 00010000:00010000:2.0:1785220581.534500:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e7e3453a/0xf62499ab92787b09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b69a expref: 358 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.534531:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920512 00000100:00100000:2.0:1785220581.534534:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920512 00010000:00010000:2.0:1785220581.534545:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e1f278f1/0xf62499ab92787cb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6fc expref: 359 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.534562:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519920768 00000100:00100000:2.0:1785220581.534564:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519920768 00010000:00010000:2.0:1785220581.534573:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000429cb9b5/0xf62499ab92787950 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5a5 expref: 360 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.534590:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100106 00000100:00100000:2.0:1785220581.534595:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+360:2406:x1871938519920896:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.534603:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.534608:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e7e3453a/0xf62499ab92787b09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b69a expref: 360 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534618:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e7e3453a/0xf62499ab92787b09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b69a expref: 360 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534632:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7e3453a/0xf62499ab92787b09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b69a expref: 360 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534643:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7206:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.534646:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7206:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.534651:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534659:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534674:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.534696:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+359:2406:x1871938519920896:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.534702:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100106 00000100:00100000:2.0:1785220581.534737:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100107 00000100:00100000:2.0:1785220581.534743:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+358:2406:x1871938519920512:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.534749:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.534753:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e1f278f1/0xf62499ab92787cb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6fc expref: 358 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534780:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e1f278f1/0xf62499ab92787cb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6fc expref: 358 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534792:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1f278f1/0xf62499ab92787cb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6fc expref: 358 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534801:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7207:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.534804:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7207:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.534808:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534815:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534830:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.534853:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+357:2406:x1871938519920512:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (791us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.534860:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100107 00000100:00100000:2.0:1785220581.534871:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100108 00000100:00100000:2.0:1785220581.534876:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+356:2406:x1871938519920768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.534883:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.534887:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000429cb9b5/0xf62499ab92787950 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5a5 expref: 356 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534902:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000429cb9b5/0xf62499ab92787950 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5a5 expref: 356 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534913:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000429cb9b5/0xf62499ab92787950 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5a5 expref: 356 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534921:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7204:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.534924:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7204:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.534928:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534935:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.534946:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.534960:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+355:2406:x1871938519920768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (883us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.534966:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100108 00000100:00100000:2.0:1785220581.534992:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ad880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510625152:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.535006:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535025:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535039:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535048:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535055:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535069:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.535205:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1125:3654:x1871938510624512:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13249us (13640us total) trans 4295070496 rc 0/0 00000100:00100000:2.0:1785220581.535325:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284175 00010000:00010000:2.0:1785220581.535346:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535354:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535362:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535370:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535386:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.535449:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1125:3652:x1871938510624256:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13801us (14041us total) trans 4295070497 rc 0/0 00000100:00100000:2.0:1785220581.535456:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284173 00010000:00010000:2.0:1785220581.535473:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535482:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535490:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535498:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535510:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.535569:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1125:3653:x1871938510624384:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13746us (14077us total) trans 4295070498 rc 0/0 00000100:00100000:2.0:1785220581.535574:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284174 00010000:00010000:2.0:1785220581.535588:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.535593:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535601:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.535605:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535614:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535625:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535633:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f740edce/0xf62499ab927a494f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535647:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.535650:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535656:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.535660:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535667:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535675:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535681:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4941 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535693:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.535696:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535702:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fee00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.535706:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535712:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535720:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535727:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000976ffad2/0xf62499ab927a4948 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.535736:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.535737:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.535739:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.535743:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.535785:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724000/2, rc 0, transno 4295070498 00000100:00100000:2.0:1785220581.535795:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510624384:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.536060:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536069:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536077:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536097:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536115:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.536208:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1125:3652:x1871938510623616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15223us (15511us total) trans 4295070499 rc 0/0 00000100:00100000:1.0:1785220581.536286:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284172 00010000:00010000:1.0:1785220581.536312:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.536317:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536324:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.536327:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536333:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536344:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536351:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a493a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.536363:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.536365:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.536366:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.536386:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b030000/2, rc 0, transno 4295070497 00000100:00100000:1.0:1785220581.536391:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510624256:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.536398:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72c700/2, rc 0, transno 4295070499 00000100:00100000:1.0:1785220581.536401:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510623616:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.536527:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.536534:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.536548:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510627456:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.536582:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.536612:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510627456 00000100:00100000:2.0:1785220581.536617:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510627456 00000100:00100000:2.0:1785220581.537348:0:14170:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284186 00000100:00100000:2.0:1785220581.537358:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+1125:3653:x1871938510627456:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.537384:0:14170:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29196 count 1 00010000:00010000:2.0:1785220581.537405:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.537422:0:14170:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000036ccb793/0xf62499ab9278803b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b91e expref: 354 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.537431:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x720c:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.537435:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.537440:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.537467:0:14170:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000036ccb793/0xf62499ab9278803b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b91e expref: 354 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.537479:0:14170:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000036ccb793/0xf62499ab9278803b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b91e expref: 354 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.537492:0:14170:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510627584:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.537541:0:14170:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.537588:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510627712:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.537617:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.537701:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.537729:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99065c00/2, rc 0, transno 4295069904 00000100:00100000:0.0:1785220581.537740:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510622720:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.537968:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510625792:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.537989:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.538061:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.538078:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921152 00000100:00100000:0.0:1785220581.538105:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921152 00010000:00010000:0.0:1785220581.538141:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e12d24d0/0xf62499ab927849f4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f63 expref: 403 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.538168:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100109 00000100:00100000:0.0:1785220581.538175:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+403:2406:x1871938519921152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.538186:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.538192:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e12d24d0/0xf62499ab927849f4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f63 expref: 403 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538202:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e12d24d0/0xf62499ab927849f4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f63 expref: 403 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538303:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.538307:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.538312:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538322:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538337:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.538363:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+403:2406:x1871938519921152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 188us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.538370:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100109 00000100:00100000:0.0:1785220581.538404:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.538437:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.538452:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921408 00000100:00100000:0.0:1785220581.538456:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921408 00010000:00010000:0.0:1785220581.538471:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c2a09df3/0xf62499ab92784c24 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a066 expref: 403 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.538492:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921024 00000100:00100000:0.0:1785220581.538495:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921024 00010000:00010000:0.0:1785220581.538508:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eb78eedc/0xf62499ab92787e97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b814 expref: 355 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.538527:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100110 00000100:00100000:0.0:1785220581.538534:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+403:2406:x1871938519921408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.538542:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.538548:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c2a09df3/0xf62499ab92784c24 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a066 expref: 403 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538557:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c2a09df3/0xf62499ab92784c24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a066 expref: 403 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538569:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.538572:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.538576:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538583:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538594:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.538612:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+403:2406:x1871938519921408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.538619:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100110 00000100:00100000:0.0:1785220581.538632:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100111 00000100:00100000:0.0:1785220581.538638:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+355:2406:x1871938519921024:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.538646:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.538651:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eb78eedc/0xf62499ab92787e97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b814 expref: 355 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538665:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eb78eedc/0xf62499ab92787e97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b814 expref: 355 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538677:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb78eedc/0xf62499ab92787e97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b814 expref: 355 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538690:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7209:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.538693:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7209:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.538697:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538705:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.538725:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.538742:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2406:x1871938519921024:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.538748:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100111 00000100:00100000:0.0:1785220581.538795:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.538817:0:14164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2a09df3/0xf62499ab92784c24 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a066 expref: 402 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.538829:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510626048:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.538845:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.538852:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.538854:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.538866:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.538875:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.538884:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.538898:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.539166:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1053:3654:x1871938510625920:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11620us (11659us total) trans 4295069905 rc 0/0 00000100:00100000:3.0:1785220581.539174:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284180 00010000:00010000:0.0:1785220581.539450:0:14156:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e12d24d0/0xf62499ab927849f4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79f63 expref: 401 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.539461:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510625536:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.539477:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.539485:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.539491:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.539497:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.539504:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.539518:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.539596:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1053:3652:x1871938510625408:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15046us (15199us total) trans 4295069906 rc 0/0 00000100:00100000:0.0:1785220581.539604:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284178 00000100:00100000:2.0:1785220581.539698:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510626304:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.539712:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539727:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.539732:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539739:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fec00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.539743:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539751:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539775:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539782:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002900884/0xf62499ab927a4972 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539801:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.539805:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539816:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.539819:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.539825:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.540037:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.540046:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4964 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.540065:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.540067:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.540069:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.540196:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.540279:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510627840:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.540317:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.541331:0:14164:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510627712 00000100:00100000:3.0:1785220581.541340:0:14164:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510627712 00010000:00010000:1.0:1785220581.541358:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541370:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541380:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.541382:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284187 00010000:00010000:1.0:1785220581.541388:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.541390:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1126:3653:x1871938510627712:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.541403:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00002000:00080000:3.0:1785220581.541420:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29197 count 1 00010000:00010000:3.0:1785220581.541445:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.541461:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ac40df48/0xf62499ab92788096 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b96b expref: 353 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.541469:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x720d:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.541473:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.541477:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.541506:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ac40df48/0xf62499ab92788096 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b96b expref: 353 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.541518:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ac40df48/0xf62499ab92788096 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b96b expref: 353 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.541531:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510627968:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.541550:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1127:3654:x1871938510624640:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18473us (18587us total) trans 4295070500 rc 0/0 00000100:00100000:1.0:1785220581.541558:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284177 00000100:00100000:1.0:1785220581.541584:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921280 00000100:00100000:1.0:1785220581.541588:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921280 00010000:00010000:1.0:1785220581.541606:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000037091e1e/0xf62499ab92784afe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79fe1 expref: 401 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.541629:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921536 00000100:00100000:1.0:1785220581.541632:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921536 00010000:00010000:1.0:1785220581.541644:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000046e646aa/0xf62499ab92784cda lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0ba expref: 402 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.541661:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100112 00000100:00100000:1.0:1785220581.541666:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+402:2406:x1871938519921280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.541675:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.541680:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000037091e1e/0xf62499ab92784afe lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79fe1 expref: 402 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541690:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000037091e1e/0xf62499ab92784afe lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79fe1 expref: 402 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541709:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000037091e1e/0xf62499ab92784afe lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79fe1 expref: 402 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541722:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.541725:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.541731:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541740:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541773:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.541805:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+401:2406:x1871938519921280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 138us (3010us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.541812:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100112 00000100:00100000:1.0:1785220581.541826:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100113 00000100:00100000:1.0:1785220581.541844:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e12050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+400:2407:x1871938519921536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.541851:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.541856:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000046e646aa/0xf62499ab92784cda lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0ba expref: 400 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541865:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000046e646aa/0xf62499ab92784cda lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0ba expref: 400 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541877:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046e646aa/0xf62499ab92784cda lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a0ba expref: 400 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541887:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.541889:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.541893:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541904:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.541920:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.541936:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e12050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+399:2407:x1871938519921536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (3084us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.541943:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100113 00010000:00010000:1.0:1785220581.541989:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.541995:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.542003:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.542007:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.542015:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.542027:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.542034:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a495d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.542052:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.542054:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.542056:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.542490:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ace00/2, rc 0, transno 4295069906 00000100:00100000:1.0:1785220581.542497:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510625408:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.542553:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.542564:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510628224:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.542595:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.542659:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510626560:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.542681:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.542707:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510627200:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.542717:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.542743:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.543000:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.543126:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510628224 00000100:00100000:0.0:1785220581.543143:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510628224 00000100:00100000:0.0:1785220581.543175:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284188 00000100:00100000:0.0:1785220581.543183:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1053:3652:x1871938510628224:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.543204:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29142 count 1 00010000:00010000:0.0:1785220581.543336:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543359:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c658292a/0xf62499ab9278505a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a242 expref: 398 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543367:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d6:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.543372:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.543376:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543405:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c658292a/0xf62499ab9278505a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a242 expref: 398 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543416:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c658292a/0xf62499ab9278505a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a242 expref: 398 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.543428:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510628352:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.543471:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.543493:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543501:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543511:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543523:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543536:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.543668:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1054:3653:x1871938510625664:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16724us (17212us total) trans 4295069907 rc 0/0 00000100:00100000:0.0:1785220581.543675:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284179 00010000:00010000:0.0:1785220581.543697:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.543702:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543710:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.543714:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543725:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543743:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543751:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a496b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.543783:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.543785:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.543787:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.544126:0:14260:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510627840 00000100:00100000:2.0:1785220581.544190:0:14260:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510627840 00000100:00100000:2.0:1785220581.544328:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284189 00000100:00100000:2.0:1785220581.544338:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90e5d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1127:3655:x1871938510627840:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.544355:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29198 count 1 00010000:00010000:2.0:1785220581.544370:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544386:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003ec1e2b5/0xf62499ab92788106 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9b1 expref: 353 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544394:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x720e:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.544398:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.544401:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544427:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003ec1e2b5/0xf62499ab92788106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9b1 expref: 353 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544438:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003ec1e2b5/0xf62499ab92788106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9b1 expref: 353 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.544450:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb36d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0000_UUID:14260:1871938510628480:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.544634:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.544654:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544663:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544672:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544887:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.544907:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.545397:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1054:3654:x1871938510626176:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16878us (16940us total) trans 4295069908 rc 0/0 00000100:00100000:2.0:1785220581.545405:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284181 00000100:00100000:2.0:1785220581.545463:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510626816:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.545484:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.545512:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510627328:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.545526:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.545544:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.545550:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.545558:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fea00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.545562:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.545572:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.545593:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.545601:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4979 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.545626:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.545629:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.545630:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.545638:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.545670:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be4700/2, rc 0, transno 4295069907 00000100:00100000:2.0:1785220581.545687:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510625664:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.545949:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510628608:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.545994:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.546035:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510628608 00000100:00100000:2.0:1785220581.546041:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510628608 00000100:00100000:2.0:1785220581.546075:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284190 00000100:00100000:2.0:1785220581.546104:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1054:3653:x1871938510628608:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.546127:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29143 count 1 00010000:00010000:2.0:1785220581.546493:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546508:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e3b9509c/0xf62499ab92785109 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a29d expref: 398 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546520:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d7:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.546524:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.546528:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546553:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e3b9509c/0xf62499ab92785109 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a29d expref: 398 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546564:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e3b9509c/0xf62499ab92785109 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a29d expref: 398 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.546577:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510628736:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.546673:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.546709:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510628096:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.546736:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.546757:0:14168:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510628096 00000100:00100000:2.0:1785220581.546770:0:14168:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510628096 00000100:00100000:2.0:1785220581.546796:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284191 00000100:00100000:2.0:1785220581.546802:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1055:3655:x1871938510628096:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.546815:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29141 count 1 00010000:00010000:2.0:1785220581.546827:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546848:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bfed7624/0xf62499ab92784f96 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1fc expref: 398 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546855:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d5:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.546857:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.546860:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546878:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bfed7624/0xf62499ab92784f96 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1fc expref: 398 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.546887:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bfed7624/0xf62499ab92784f96 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1fc expref: 398 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.546897:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510628864:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.547024:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.547573:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.547601:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921920 00000100:00100000:3.0:1785220581.547607:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921920 00010000:00010000:3.0:1785220581.547633:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000df6dcb55/0xf62499ab92784e62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a17e expref: 399 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.547669:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922176 00000100:00100000:3.0:1785220581.547673:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922176 00010000:00010000:3.0:1785220581.547688:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007dd522b6/0xf62499ab92787fc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8df expref: 354 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.547717:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100114 00000100:00100000:3.0:1785220581.547726:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+399:2407:x1871938519921920:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.547735:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.547740:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000df6dcb55/0xf62499ab92784e62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a17e expref: 399 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.547752:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000df6dcb55/0xf62499ab92784e62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a17e expref: 399 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.547822:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df6dcb55/0xf62499ab92784e62 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a17e expref: 399 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.547837:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.547840:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.547845:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.547855:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.547882:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.547910:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100115 00000100:00100000:3.0:1785220581.547912:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+398:2407:x1871938519921920:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 189us (5170us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.547919:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100114 00000100:00100000:0.0:1785220581.547920:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2407:x1871938519922176:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.547928:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.547933:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007dd522b6/0xf62499ab92787fc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8df expref: 354 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.547943:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007dd522b6/0xf62499ab92787fc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8df expref: 354 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.547958:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dd522b6/0xf62499ab92787fc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8df expref: 354 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.547969:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.547972:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.547976:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.547984:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.548006:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.548025:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2407:x1871938519922176:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (5026us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.548032:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100115 00010000:00010000:2.0:1785220581.548372:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.548383:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.548392:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.548400:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.548461:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.548622:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1056:3652:x1871938510626688:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18902us (18985us total) trans 4295069909 rc 0/0 00000100:00100000:2.0:1785220581.548631:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284183 00010000:00010000:2.0:1785220581.548653:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.548663:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.548671:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.548679:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.548777:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.548783:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.548792:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.548796:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.548805:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.548818:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.548825:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4987 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.549035:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.549040:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.549047:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.549049:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.549056:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.549071:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.549079:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.549081:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.549115:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.549422:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1785220581.549428:0:14163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a96fe265/0xf62499ab927a4995 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.549559:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1128:3654:x1871938510627072:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19109us (19157us total) trans 4295070501 rc 0/0 00000100:00100000:2.0:1785220581.549566:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284185 00000004:00080000:2.0:1785220581.549615:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99066680/2, rc 0, transno 4295069909 00000100:00100000:2.0:1785220581.549635:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99066680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510626688:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.549684:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a5880/2, rc 0, transno 4295070501 00000100:00100000:2.0:1785220581.549696:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510627072:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.549754:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.549772:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510628992:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.549808:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.549852:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99066d80/2, rc 0, transno 4295069908 00000100:00100000:2.0:1785220581.549857:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99066d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510626176:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.549866:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be5c00/2, rc 0, transno 4295069905 00000100:00100000:2.0:1785220581.549869:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510625920:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.549878:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725180/2, rc 0, transno 4295070500 00000100:00100000:2.0:1785220581.549885:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510624640:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.549914:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998bbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510629120:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.549939:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.549971:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f726300/2, rc 0, transno 4295070496 00000100:00100000:2.0:1785220581.549977:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510624512:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.550007:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510629248:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.550023:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921792 00000100:00100000:2.0:1785220581.550029:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.550030:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921792 00000100:00100000:2.0:1785220581.550057:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510628992 00010000:00010000:3.0:1785220581.550058:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fa95305a/0xf62499ab92784d89 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a12a expref: 398 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.550060:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510628992 00000100:00100000:2.0:1785220581.550081:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510629120 00000100:00100000:2.0:1785220581.550105:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510629120 00000100:00100000:2.0:1785220581.550127:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510629248 00000100:00100000:2.0:1785220581.550136:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510629248 00000100:00100000:2.0:1785220581.550158:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284192 00000100:00100000:3.0:1785220581.550165:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100116 00000100:00100000:2.0:1785220581.550166:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1130:3655:x1871938510628992:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.550171:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d977f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+398:2407:x1871938519921792:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.550181:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00080000:2.0:1785220581.550185:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29199 count 1 00010000:00010000:3.0:1785220581.550187:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fa95305a/0xf62499ab92784d89 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a12a expref: 398 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550197:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fa95305a/0xf62499ab92784d89 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a12a expref: 398 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.550206:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550217:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa95305a/0xf62499ab92784d89 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a12a expref: 398 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550229:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.550234:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.550239:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550249:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550271:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.550300:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d977f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2407:x1871938519921792:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 129us (2728us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.550307:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100116 00000100:00100000:3.0:1785220581.550331:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284193 00000100:00100000:3.0:1785220581.550338:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e05180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1130:3655:x1871938510629120:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.550343:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e08876b2/0xf62499ab927881f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba1a expref: 352 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.550350:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29200 count 1 00010000:00010000:2.0:1785220581.550350:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x720f:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.550353:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.550357:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550362:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550372:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002392417e/0xf62499ab927882db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba9f expref: 352 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550378:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7210:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.550378:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e08876b2/0xf62499ab927881f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba1a expref: 352 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550381:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7210:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.550385:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.550390:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e08876b2/0xf62499ab927881f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba1a expref: 352 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.550398:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002392417e/0xf62499ab927882db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba9f expref: 352 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.550402:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510629376:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.550407:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002392417e/0xf62499ab927882db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba9f expref: 352 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.550416:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510629504:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.550436:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.550436:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.550449:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.550450:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284194 00010000:00010000:3.0:1785220581.550457:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.550457:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1130:3652:x1871938510629248:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.550467:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.550469:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29201 count 1 00010000:00010000:3.0:1785220581.550475:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.550479:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.550521:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003082ffe9/0xf62499ab927883b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb0f expref: 352 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.550528:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7211:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.550531:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7211:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.550633:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.550651:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003082ffe9/0xf62499ab927883b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb0f expref: 352 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.550659:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003082ffe9/0xf62499ab927883b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb0f expref: 352 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.550669:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510630016:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.550687:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.550704:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220581.550711:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510629632:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.550732:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.550781:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510629888:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.550800:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.550815:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.550819:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510629760:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.550997:0:13874:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510629632 00000100:00100000:1.0:1785220581.551002:0:13874:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510629632 00000100:00100000:1.0:1785220581.551030:0:13874:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510629888 00000100:00100000:1.0:1785220581.551032:0:13874:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510629888 00000100:00100000:1.0:1785220581.551049:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284195 00000100:00100000:1.0:1785220581.551053:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1057:3652:x1871938510629632:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.551064:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29144 count 1 00010000:00010000:1.0:1785220581.551075:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.551115:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000332301be/0xf62499ab92785213 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a322 expref: 396 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.551124:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d8:0x0].0x0 since=31s v=(0 0) 00010000:00010000:1.0:1785220581.551126:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.551130:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.551150:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000332301be/0xf62499ab92785213 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a322 expref: 396 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.551159:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000332301be/0xf62499ab92785213 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a322 expref: 396 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.551170:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510630144:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.551197:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df019c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.551457:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284196 00000100:00100000:2.0:1785220581.551465:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1057:3655:x1871938510629888:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.551477:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29146 count 1 00010000:00010000:2.0:1785220581.551487:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.551496:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005aaee8da/0xf62499ab92785363 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3d8 expref: 396 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.551500:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71da:0x0].0x0 since=31s v=(0 0) 00010000:00010000:2.0:1785220581.551502:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.551504:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.551517:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005aaee8da/0xf62499ab92785363 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3d8 expref: 396 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.551523:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005aaee8da/0xf62499ab92785363 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3d8 expref: 396 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.551529:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510630272:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.551580:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.551610:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.551644:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbd500/2, rc 0, transno 4295069903 00000100:00100000:2.0:1785220581.551651:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510623744:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.551721:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.551728:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551736:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.551740:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551749:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551773:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551782:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.551784:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.551786:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.551798:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510629760 00000100:00100000:0.0:1785220581.551801:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510629760 00010000:00010000:3.0:1785220581.551818:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220581.551824:0:14256:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4980 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.551827:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284197 00000100:00100000:0.0:1785220581.551894:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1058:3655:x1871938510629760:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.551907:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29145 count 1 00010000:00010000:0.0:1785220581.551918:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551929:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000019c0d88e/0xf62499ab927852de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3a0 expref: 396 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551935:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71d9:0x0].0x0 since=31s v=(0 0) 00010000:00010000:0.0:1785220581.551938:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.551941:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551959:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000019c0d88e/0xf62499ab927852de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3a0 expref: 396 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.551969:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000019c0d88e/0xf62499ab927852de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3a0 expref: 396 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.551973:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1059:3652:x1871938510626432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22892us (22971us total) trans 4295069910 rc 0/0 00000100:00100000:0.0:1785220581.551979:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510630400:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.551980:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284182 00000100:00100000:0.0:1785220581.551998:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.552043:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.552064:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99067800/2, rc 0, transno 4295069910 00000100:00100000:2.0:1785220581.552069:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99067800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510626432:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.552573:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.552604:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.552622:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922048 00000100:00100000:0.0:1785220581.552627:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922048 00010000:00010000:0.0:1785220581.552650:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000053d0b2b9/0xf62499ab92787f3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b884 expref: 353 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.552672:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519921664 00000100:00100000:0.0:1785220581.552675:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519921664 00010000:00010000:0.0:1785220581.552688:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e79bb9e7/0xf62499ab92787a0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b61c expref: 354 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.552708:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100117 00000100:00100000:0.0:1785220581.552715:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2407:x1871938519922048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.552726:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.552731:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000053d0b2b9/0xf62499ab92787f3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b884 expref: 354 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.552741:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000053d0b2b9/0xf62499ab92787f3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b884 expref: 354 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.552758:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053d0b2b9/0xf62499ab92787f3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b884 expref: 354 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.552784:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.552787:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.552792:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.552799:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100118 00010000:00010000:0.0:1785220581.552801:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.552808:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2405:x1871938519921664:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.552816:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.552821:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e79bb9e7/0xf62499ab92787a0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b61c expref: 353 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.552827:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.552830:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e79bb9e7/0xf62499ab92787a0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b61c expref: 353 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552843:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e79bb9e7/0xf62499ab92787a0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b61c expref: 353 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552854:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7205:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.552857:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7205:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.552861:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552869:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552886:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.552912:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+352:2405:x1871938519921664:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.552919:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100118 00010000:00010000:2.0:1785220581.552922:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.552931:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552937:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.552941:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552946:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.552950:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552959:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.552964:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.552966:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.552976:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.553073:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1132:3654:x1871938510626944:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22720us (22774us total) trans 4295070503 rc 0/0 00000100:00100000:3.0:1785220581.553080:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284184 00000100:00100000:1.0:1785220581.553151:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510630528:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.553183:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.553234:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510630528 00000100:00100000:3.0:1785220581.553238:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510630528 00000100:00100000:3.0:1785220581.553264:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284198 00000100:00100000:3.0:1785220581.553270:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1059:3652:x1871938510630528:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.553285:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29147 count 1 00010000:00010000:3.0:1785220581.553301:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.553314:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000081630f25/0xf62499ab927853b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a410 expref: 396 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.553320:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71db:0x0].0x0 since=31s v=(0 0) 00010000:00010000:3.0:1785220581.553323:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.553327:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.553353:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000081630f25/0xf62499ab927853b0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a410 expref: 396 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.553363:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000081630f25/0xf62499ab927853b0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a410 expref: 396 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.553370:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+351:2407:x1871938519922048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 655us (798us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.553374:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510630656:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.553377:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100117 00010000:00010000:0.0:1785220581.553397:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.553402:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.553405:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.553410:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.553414:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553424:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553436:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553443:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081ef1dad/0xf62499ab927a4956 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553458:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.553461:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553471:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.553474:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553481:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553490:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553497:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a498e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.553511:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.553513:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.553515:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.553585:0:14170:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510627584:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.553586:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.553605:0:14170:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.553628:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510627968:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.553639:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.553679:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.553718:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510628352:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.553734:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.553776:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.553824:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.553883:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb36d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0000_UUID:14260:1871938510628480:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.553899:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.553907:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922304 00000100:00100000:1.0:1785220581.553911:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922304 00010000:00010000:1.0:1785220581.553929:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000036ccb793/0xf62499ab9278803b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b91e expref: 351 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.553949:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922432 00000100:00100000:1.0:1785220581.553951:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922432 00010000:00010000:1.0:1785220581.553964:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ac40df48/0xf62499ab92788096 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b96b expref: 352 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.553989:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922560 00000100:00100000:1.0:1785220581.553992:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922560 00010000:00010000:1.0:1785220581.554006:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c658292a/0xf62499ab9278505a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a242 expref: 397 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.554023:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922688 00000100:00100000:1.0:1785220581.554026:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922688 00010000:00010000:1.0:1785220581.554038:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003ec1e2b5/0xf62499ab92788106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9b1 expref: 353 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.554042:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100119 00000100:00100000:3.0:1785220581.554050:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2406:x1871938519922304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.554056:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100120 00010000:00010000:3.0:1785220581.554059:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220581.554061:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2407:x1871938519922432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.554064:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000036ccb793/0xf62499ab9278803b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b91e expref: 353 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.554067:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.554072:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ac40df48/0xf62499ab92788096 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b96b expref: 353 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554074:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000036ccb793/0xf62499ab9278803b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b91e expref: 353 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.554081:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ac40df48/0xf62499ab92788096 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b96b expref: 353 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554107:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036ccb793/0xf62499ab9278803b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b91e expref: 353 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.554109:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ac40df48/0xf62499ab92788096 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b96b expref: 353 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554119:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.554119:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.554123:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.554123:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.554128:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.554128:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554136:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.554148:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554153:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.554163:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.554632:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.554659:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+351:2406:x1871938519922304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 609us (1072us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.554667:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100119 00000100:00100000:3.0:1785220581.554685:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922944 00000100:00100000:3.0:1785220581.554688:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922944 00010000:00010000:3.0:1785220581.554703:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bfed7624/0xf62499ab92784f96 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1fc expref: 398 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.554728:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100121 00000100:00100000:3.0:1785220581.554734:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+398:2406:x1871938519922560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.554741:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220581.554743:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.554746:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c658292a/0xf62499ab9278505a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a242 expref: 398 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554757:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c658292a/0xf62499ab9278505a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a242 expref: 398 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.554758:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519922816 00000100:00100000:0.0:1785220581.554773:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519922816 00010000:00010000:3.0:1785220581.554784:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c658292a/0xf62499ab9278505a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a242 expref: 399 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.554788:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e3b9509c/0xf62499ab92785109 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a29d expref: 399 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554796:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.554800:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.554804:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.554808:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100122 00010000:00010000:3.0:1785220581.554812:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.554814:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+350:2406:x1871938519922688:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.554822:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.554826:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003ec1e2b5/0xf62499ab92788106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9b1 expref: 350 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554829:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.554846:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+398:2406:x1871938519922560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (1071us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.554849:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003ec1e2b5/0xf62499ab92788106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9b1 expref: 350 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.554852:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100121 00010000:00010000:0.0:1785220581.554862:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ec1e2b5/0xf62499ab92788106 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9b1 expref: 350 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.554866:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100123 00000100:00100000:3.0:1785220581.554872:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2407:x1871938519922944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.554873:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.554876:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.554879:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.554880:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554883:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bfed7624/0xf62499ab92784f96 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1fc expref: 397 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.554889:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554893:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bfed7624/0xf62499ab92784f96 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1fc expref: 397 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554905:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.554908:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.554911:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.554912:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554919:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.554927:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+349:2406:x1871938519922688:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (1103us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.554930:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.554933:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100122 00000100:00100000:3.0:1785220581.554943:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2407:x1871938519922944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 73us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.554946:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100124 00000100:00100000:3.0:1785220581.554950:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100123 00000100:00100000:0.0:1785220581.554952:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2407:x1871938519922816:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.554960:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.554964:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e3b9509c/0xf62499ab92785109 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a29d expref: 396 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554968:0:14170:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.554974:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e3b9509c/0xf62499ab92785109 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a29d expref: 396 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554976:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.554985:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.554988:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.554990:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.554992:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.554999:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555000:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555012:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.555014:0:14170:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.555028:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+396:2407:x1871938519922816:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.555037:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100124 00010000:00010000:0.0:1785220581.555056:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555063:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555070:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555077:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555113:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.555278:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+1132:3653:x1871938510627456:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17922us (18694us total) trans 4295070504 rc 0/0 00000100:00100000:3.0:1785220581.555286:0:14170:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284186 00010000:00010000:3.0:1785220581.555312:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.555321:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.555329:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.555336:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.555350:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.555391:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1060:3652:x1871938510628224:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12208us (12794us total) trans 4295069911 rc 0/0 00000100:00100000:0.0:1785220581.555398:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284188 00010000:00010000:0.0:1785220581.555415:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.555421:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.555425:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90e5d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1132:3655:x1871938510627840:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11090us (15108us total) trans 4295070505 rc 0/0 00010000:00010000:0.0:1785220581.555430:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffe00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:3.0:1785220581.555434:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284189 00010000:00010000:0.0:1785220581.555434:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555443:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555455:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555462:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000663e34e8/0xf62499ab927a499c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555479:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.555483:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555490:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.555493:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555501:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555511:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555518:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a49aa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555531:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.555535:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555542:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.555545:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555552:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555562:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555569:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000998b8cac/0xf62499ab927a49b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.555583:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.555584:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.555586:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.555593:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.556009:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+348:2407:x1871938519922432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1944us (2326us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.556018:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100120 00010000:00010000:1.0:1785220581.556068:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.556076:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.556106:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.556114:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.556126:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.556352:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.556357:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.556364:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.556367:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.556378:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.556389:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.556396:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a49a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.556414:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.556415:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.556417:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.558626:0:14168:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bfed7624/0xf62499ab92784f96 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a1fc expref: 395 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.558652:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510628864:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.558677:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.558688:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.558696:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.558708:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.558708:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.558715:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.558726:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.558733:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.558747:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923200 00000100:00100000:0.0:1785220581.558753:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923200 00010000:00010000:0.0:1785220581.558801:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002392417e/0xf62499ab927882db lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba9f expref: 348 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.558829:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923072 00000100:00100000:0.0:1785220581.558831:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923072 00010000:00010000:0.0:1785220581.558855:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e08876b2/0xf62499ab927881f4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba1a expref: 349 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.558878:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100125 00000100:00100000:0.0:1785220581.558886:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+349:2407:x1871938519923200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.558902:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.558907:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002392417e/0xf62499ab927882db lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba9f expref: 349 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.558916:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1060:3655:x1871938510628096:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12113us (12179us total) trans 4295069912 rc 0/0 00010000:00010000:0.0:1785220581.558918:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002392417e/0xf62499ab927882db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba9f expref: 349 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.558924:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284191 00010000:00010000:0.0:1785220581.558942:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7210:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.558947:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7210:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.558952:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.558962:0:13867:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e3b9509c/0xf62499ab92785109 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a29d expref: 394 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.558962:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.558973:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510628736:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.558984:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.558984:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.558992:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.558998:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559006:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.559010:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+349:2407:x1871938519923200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 125us (302us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.559014:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.559016:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100125 00010000:00010000:3.0:1785220581.559027:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.559029:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100126 00000100:00100000:0.0:1785220581.559035:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+348:2407:x1871938519923072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.559042:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.559047:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e08876b2/0xf62499ab927881f4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba1a expref: 348 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559063:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e08876b2/0xf62499ab927881f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba1a expref: 348 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559075:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.559078:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x720f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.559082:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559109:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.559111:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1060:3653:x1871938510628608:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13029us (13117us total) trans 4295069913 rc 0/0 00000100:00100000:3.0:1785220581.559118:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284190 00010000:00010000:0.0:1785220581.559122:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.559152:0:14189:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002392417e/0xf62499ab927882db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba9f expref: 348 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.559152:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+348:2407:x1871938519923072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.559159:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100126 00000100:00100000:3.0:1785220581.559163:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510629504:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.559174:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559180:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.559182:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559186:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559189:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559194:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.559198:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559198:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559206:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559208:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559214:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.559296:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e05180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1133:3655:x1871938510629120:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8958us (9356us total) trans 4295070507 rc 0/0 00000100:00100000:3.0:1785220581.559302:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284193 00010000:00010000:3.0:1785220581.559322:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.559327:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559334:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.559337:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559349:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559362:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559370:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3137545/0xf62499ab927a49b8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559388:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.559392:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559397:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559398:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.559401:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559404:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e680526c/0xf62499ab927a49bf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559409:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.559417:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559419:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.559421:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.559422:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.559425:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a49cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.559429:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.559445:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.559447:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.559448:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.559474:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be5f80/2, rc 0, transno 4295070503 00000100:00100000:3.0:1785220581.559486:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510626944:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.559863:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1132:3653:x1871938510627712:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18470us (22242us total) trans 4295070506 rc 0/0 00000100:00100000:1.0:1785220581.559872:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284187 00000004:00080000:1.0:1785220581.559935:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99064380/2, rc 0, transno 4295069911 00000100:00100000:1.0:1785220581.559941:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510628224:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.559967:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98adf480/2, rc 0, transno 4295069913 00000100:00100000:1.0:1785220581.559971:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510628608:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.559981:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724380/2, rc 0, transno 4295070506 00000100:00100000:1.0:1785220581.559984:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510627712:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.559992:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ad880/2, rc 0, transno 4295070504 00000100:00100000:1.0:1785220581.559995:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510627456:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.560079:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.560104:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510630784:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.560421:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.560494:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510630784 00000100:00100000:3.0:1785220581.560500:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510630784 00000100:00100000:1.0:1785220581.560518:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510630912:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.560543:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.560550:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510630912 00000100:00100000:3.0:1785220581.560553:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510630912 00000100:00100000:1.0:1785220581.560566:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284199 00000100:00100000:3.0:1785220581.560572:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284200 00000100:00100000:1.0:1785220581.560573:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1061:3652:x1871938510630784:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.560577:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1061:3654:x1871938510630912:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.560596:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29149 count 1 00002000:00080000:1.0:1785220581.560596:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29148 count 1 00010000:00010000:3.0:1785220581.560619:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.560619:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.560638:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005542ab08/0xf62499ab927855d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a521 expref: 393 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.560638:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000062c09318/0xf62499ab927854cf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a495 expref: 393 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.560647:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71dd:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.560647:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71dc:0x0].0x0 since=30s v=(0 0) 00010000:00010000:3.0:1785220581.560652:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.560652:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.560657:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.560657:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.560681:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005542ab08/0xf62499ab927855d9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a521 expref: 393 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.560681:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000062c09318/0xf62499ab927854cf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a495 expref: 393 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.560694:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000062c09318/0xf62499ab927854cf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a495 expref: 393 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.560695:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005542ab08/0xf62499ab927855d9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a521 expref: 393 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.560707:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510631296:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.560707:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d977f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510631168:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.560815:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.560835:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.560852:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510631040:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.561612:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.561612:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.561644:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510631040 00000100:00100000:1.0:1785220581.561651:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510631040 00000100:00100000:1.0:1785220581.561695:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284201 00000100:00100000:1.0:1785220581.561703:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfa5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1062:3654:x1871938510631040:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.561724:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29150 count 1 00010000:00010000:1.0:1785220581.561742:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.561757:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000732e6d4a/0xf62499ab927856c7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a591 expref: 393 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.561776:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71de:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.561779:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.561783:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.561809:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000732e6d4a/0xf62499ab927856c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a591 expref: 393 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.561819:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000732e6d4a/0xf62499ab927856c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a591 expref: 393 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.561843:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfa5a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510631424:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.561871:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.562553:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.562601:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d998bbb80/2, rc 0, transno 4295070507 00000100:00100000:0.0:1785220581.562608:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998bbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510629120:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.562622:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99065180/2, rc 0, transno 4295069912 00000100:00100000:0.0:1785220581.562638:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510628096:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.562649:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f726680/2, rc 0, transno 4295070505 00000100:00100000:0.0:1785220581.562653:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510627840:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.562716:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.562780:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.562799:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923328 00000100:00100000:0.0:1785220581.562804:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923328 00010000:00010000:0.0:1785220581.562832:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003082ffe9/0xf62499ab927883b4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb0f expref: 347 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.562956:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d985e8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510631552:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.562990:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.563071:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923456 00000100:00100000:0.0:1785220581.563075:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923456 00010000:00010000:0.0:1785220581.563118:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005aaee8da/0xf62499ab92785363 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3d8 expref: 394 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.563318:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100127 00000100:00100000:0.0:1785220581.563328:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ac050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+347:2406:x1871938519923328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.563344:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.563350:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003082ffe9/0xf62499ab927883b4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb0f expref: 347 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563363:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003082ffe9/0xf62499ab927883b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb0f expref: 347 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563386:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7211:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.563391:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7211:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.563397:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563407:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563428:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.563463:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ac050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+347:2406:x1871938519923328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 137us (747us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.563470:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100127 00000100:00100000:0.0:1785220581.563507:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100128 00000100:00100000:0.0:1785220581.563514:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+394:2407:x1871938519923456:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.563522:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.563527:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005aaee8da/0xf62499ab92785363 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3d8 expref: 394 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563536:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005aaee8da/0xf62499ab92785363 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3d8 expref: 394 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563550:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.563553:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.563557:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563564:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.563577:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.563594:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+394:2407:x1871938519923456:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 81us (814us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.563600:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100128 00000100:00100000:2.0:1785220581.563704:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1131:3654:x1871938510624128:12345-0@lo:6:osp-syn-0-0.0 Request processed in 41510us (41568us total) trans 4295070502 rc 0/0 00000100:00100000:2.0:1785220581.563712:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284176 00000100:00100000:2.0:1785220581.563802:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510631552 00000100:00100000:2.0:1785220581.563807:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510631552 00000100:00100000:2.0:1785220581.563900:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284202 00000100:00100000:2.0:1785220581.563909:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1063:3652:x1871938510631552:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.563930:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29151 count 1 00010000:00010000:2.0:1785220581.563950:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.563965:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000043acaf2b/0xf62499ab9278575a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5e5 expref: 393 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.563978:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71df:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.563981:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.563984:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564009:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000043acaf2b/0xf62499ab9278575a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5e5 expref: 393 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564021:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000043acaf2b/0xf62499ab9278575a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5e5 expref: 393 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.564032:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510631680:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.564070:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.564424:0:14158:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005aaee8da/0xf62499ab92785363 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3d8 expref: 393 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.564439:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510630272:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.564456:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564465:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564476:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564485:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564493:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564513:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.564684:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1064:3655:x1871938510629888:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13219us (13881us total) trans 4295069914 rc 0/0 00000100:00100000:2.0:1785220581.564692:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284196 00010000:00010000:2.0:1785220581.564749:0:13819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003082ffe9/0xf62499ab927883b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb0f expref: 346 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.564785:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510630016:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.564797:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564805:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564810:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564819:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.564826:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.564953:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72e300/2, rc 0, transno 4295070502 00000100:00100000:1.0:1785220581.564959:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510624128:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.565026:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.565160:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1131:3652:x1871938510629248:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14703us (15128us total) trans 4295070508 rc 0/0 00000100:00100000:2.0:1785220581.565167:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284194 00010000:00010000:2.0:1785220581.565194:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.565201:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565283:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.565289:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565299:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565317:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565324:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4e9c3a/0xf62499ab927a49e2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565345:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.565349:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565356:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffa00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.565360:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565368:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565379:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565386:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b81a2b6/0xf62499ab927a49d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565400:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.565402:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.565404:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.565428:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e08876b2/0xf62499ab927881f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba1a expref: 345 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.565438:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510629376:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.565451:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565460:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565467:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565475:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565483:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.565501:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.565582:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1131:3655:x1871938510628992:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15415us (15772us total) trans 4295070509 rc 0/0 00000100:00100000:2.0:1785220581.565588:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284192 00000100:00100000:2.0:1785220581.565661:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510631808:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.565693:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.565777:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.565785:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510632192:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.565807:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.565850:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.565856:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.565864:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.565867:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.565875:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.565888:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.565895:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000767f4170/0xf62499ab927a49c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.565910:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.565912:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.565914:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.565919:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510632320:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.565942:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.565974:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510631936:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.565988:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.565997:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510632448:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.566007:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.566018:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.566020:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510632064:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.566031:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.566415:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb2300/2, rc 0, transno 4295069914 00000100:00100000:2.0:1785220581.566431:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510629888:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.566445:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72c380/2, rc 0, transno 4295070509 00000100:00100000:2.0:1785220581.566452:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510628992:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.566479:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.566486:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91479880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510632704:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.566510:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.566531:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510632576:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.566546:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.566955:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.566978:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.566996:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.567037:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.567076:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.567083:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923584 00000100:00100000:2.0:1785220581.567116:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.567206:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510631808 00000100:00100000:1.0:1785220581.567282:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510631808 00000100:00100000:1.0:1785220581.567330:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510632192 00000100:00100000:1.0:1785220581.567334:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510632192 00000100:00100000:1.0:1785220581.567353:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510632320 00000100:00100000:1.0:1785220581.567356:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510632320 00000100:00100000:1.0:1785220581.567371:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510631936 00000100:00100000:1.0:1785220581.567374:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510631936 00000100:00100000:1.0:1785220581.567387:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510632448 00000100:00100000:1.0:1785220581.567389:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510632448 00000100:00100000:1.0:1785220581.567404:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510632064 00000100:00100000:1.0:1785220581.567407:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510632064 00000100:00100000:1.0:1785220581.567424:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510632704 00000100:00100000:1.0:1785220581.567427:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510632704 00000100:00100000:1.0:1785220581.567442:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510632576 00000100:00100000:1.0:1785220581.567445:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510632576 00000100:00100000:1.0:1785220581.567465:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284203 00000100:00100000:1.0:1785220581.567473:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+1137:3653:x1871938510631808:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.567501:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29202 count 1 00010000:00010000:1.0:1785220581.567524:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567543:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000082297f5b/0xf62499ab92788440 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb78 expref: 344 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567552:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7212:0x0].0x0 since=28s v=(0 0) 00010000:00010000:1.0:1785220581.567557:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7212:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.567562:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567593:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000082297f5b/0xf62499ab92788440 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb78 expref: 344 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567605:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000082297f5b/0xf62499ab92788440 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb78 expref: 344 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.567617:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510632832:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.567675:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.567693:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284204 00000100:00100000:1.0:1785220581.567700:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1137:3653:x1871938510632192:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.567712:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29205 count 1 00010000:00010000:1.0:1785220581.567723:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567733:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000971d3fc2/0xf62499ab927886cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bceb expref: 344 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567740:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7215:0x0].0x0 since=28s v=(0 0) 00010000:00010000:1.0:1785220581.567742:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7215:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.567745:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567781:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000971d3fc2/0xf62499ab927886cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bceb expref: 344 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567791:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000971d3fc2/0xf62499ab927886cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bceb expref: 344 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.567801:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510632960:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.567823:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.567904:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284205 00000100:00100000:1.0:1785220581.567912:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1137:3653:x1871938510632320:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.567924:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29206 count 1 00010000:00010000:1.0:1785220581.567935:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567945:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000536dec88/0xf62499ab92788796 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd5b expref: 344 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567952:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7216:0x0].0x0 since=28s v=(0 0) 00010000:00010000:1.0:1785220581.567955:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7216:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.567958:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567975:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000536dec88/0xf62499ab92788796 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd5b expref: 344 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.567990:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000536dec88/0xf62499ab92788796 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd5b expref: 344 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.568000:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510633088:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.568023:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.568053:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510630144:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.568076:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.568123:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923840 00000100:00100000:1.0:1785220581.568127:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923840 00010000:00010000:1.0:1785220581.568148:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000062c09318/0xf62499ab927854cf lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a495 expref: 393 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.568170:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923712 00000100:00100000:1.0:1785220581.568173:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923712 00010000:00010000:1.0:1785220581.568185:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000019c0d88e/0xf62499ab927852de lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3a0 expref: 394 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.568201:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519923968 00000100:00100000:1.0:1785220581.568204:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923968 00010000:00010000:1.0:1785220581.568294:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000081630f25/0xf62499ab927853b0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a410 expref: 395 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.568315:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924096 00000100:00100000:1.0:1785220581.568318:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924096 00010000:00010000:1.0:1785220581.568331:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005542ab08/0xf62499ab927855d9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a521 expref: 396 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.568347:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924224 00000100:00100000:1.0:1785220581.568350:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924224 00010000:00010000:1.0:1785220581.568361:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000732e6d4a/0xf62499ab927856c7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a591 expref: 397 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.568382:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100129 00000100:00100000:1.0:1785220581.568389:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2406:x1871938519923840:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.568398:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.568404:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000062c09318/0xf62499ab927854cf lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a495 expref: 397 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568414:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000062c09318/0xf62499ab927854cf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a495 expref: 397 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568434:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.568438:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.568443:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568451:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568466:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.568491:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+397:2406:x1871938519923840:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (1511us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.568499:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100129 00000100:00100000:1.0:1785220581.568520:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100130 00000100:00100000:1.0:1785220581.568527:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+396:2406:x1871938519923712:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.568534:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.568539:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000019c0d88e/0xf62499ab927852de lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3a0 expref: 396 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568549:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000019c0d88e/0xf62499ab927852de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3a0 expref: 396 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568563:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.568566:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.568570:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568578:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568591:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.568608:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+396:2406:x1871938519923712:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (1611us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.568614:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100130 00000100:00100000:1.0:1785220581.568628:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100131 00000100:00100000:1.0:1785220581.568635:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+395:2406:x1871938519923968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.568643:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.568647:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000081630f25/0xf62499ab927853b0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a410 expref: 395 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568656:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000081630f25/0xf62499ab927853b0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a410 expref: 395 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568667:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.568670:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.568674:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568681:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.568692:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.568711:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+395:2406:x1871938519923968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (1672us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.568717:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100131 00000100:00100000:1.0:1785220581.568730:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100132 00000100:00100000:1.0:1785220581.568736:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+394:2406:x1871938519924096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.568743:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.568748:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005542ab08/0xf62499ab927855d9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a521 expref: 394 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.568986:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519923584 00010000:00010000:3.0:1785220581.569010:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000332301be/0xf62499ab92785213 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a322 expref: 395 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.569036:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100133 00000100:00100000:3.0:1785220581.569043:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+395:2406:x1871938519924224:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.569050:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.569054:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000732e6d4a/0xf62499ab927856c7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a591 expref: 395 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569064:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000732e6d4a/0xf62499ab927856c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a591 expref: 395 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569079:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.569081:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.569109:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569117:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569132:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.569159:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+395:2406:x1871938519924224:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 117us (2042us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.569165:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284206 00000100:00100000:3.0:1785220581.569166:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100133 00000100:00100000:2.0:1785220581.569174:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1137:3654:x1871938510631936:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.569177:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100134 00000100:00100000:3.0:1785220581.569182:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+394:2406:x1871938519923584:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.569188:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00080000:2.0:1785220581.569190:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29203 count 1 00010000:00010000:3.0:1785220581.569193:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000332301be/0xf62499ab92785213 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a322 expref: 394 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569201:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000332301be/0xf62499ab92785213 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a322 expref: 394 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.569205:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569213:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000332301be/0xf62499ab92785213 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a322 expref: 394 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569227:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.569230:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.569234:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569240:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569257:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.569275:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+393:2406:x1871938519923584:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (2321us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.569281:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100134 00010000:00010000:2.0:1785220581.569296:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ec85ab24/0xf62499ab92788504 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bbd3 expref: 344 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.569303:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7213:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.569306:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7213:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.569309:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.569318:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510631296:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.569331:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ec85ab24/0xf62499ab92788504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bbd3 expref: 344 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569332:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.569343:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ec85ab24/0xf62499ab92788504 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bbd3 expref: 344 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.569355:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510633216:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.569356:0:14187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000081630f25/0xf62499ab927853b0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a410 expref: 392 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.569365:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510630656:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.569374:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569382:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.569387:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.569390:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569399:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569406:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569430:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.569460:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510631680:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.569474:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.569569:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1065:3652:x1871938510630528:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16298us (16384us total) trans 4295069915 rc 0/0 00000100:00100000:3.0:1785220581.569576:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284198 00000100:00100000:3.0:1785220581.569592:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284207 00000100:00100000:3.0:1785220581.569600:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1137:3654:x1871938510632448:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.569613:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29207 count 1 00010000:00010000:3.0:1785220581.569627:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569639:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000797cd97f/0xf62499ab927888fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bdf5 expref: 344 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569646:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7217:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.569648:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7217:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.569696:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.569696:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.569715:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000797cd97f/0xf62499ab927888fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bdf5 expref: 344 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569725:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000797cd97f/0xf62499ab927888fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bdf5 expref: 344 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.569736:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510633344:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.569758:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.569785:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924352 00000100:00100000:3.0:1785220581.569790:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924352 00010000:00010000:3.0:1785220581.569806:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000043acaf2b/0xf62499ab9278575a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5e5 expref: 392 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.569834:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100135 00000100:00100000:3.0:1785220581.569842:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+392:2406:x1871938519924352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.569850:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.569854:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000043acaf2b/0xf62499ab9278575a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5e5 expref: 392 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569865:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000043acaf2b/0xf62499ab9278575a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5e5 expref: 392 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569918:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000043acaf2b/0xf62499ab9278575a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a5e5 expref: 392 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569928:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.569932:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.569936:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569943:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.569967:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.569987:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.569991:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+391:2406:x1871938519924352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 150us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.569997:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100135 00010000:00010000:2.0:1785220581.570001:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.570010:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.570017:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.570038:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.570148:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1065:3652:x1871938510631552:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6241us (7158us total) trans 4295069916 rc 0/0 00000100:00100000:2.0:1785220581.570156:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284202 00000100:00100000:2.0:1785220581.570173:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284208 00000100:00100000:2.0:1785220581.570179:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1137:3654:x1871938510632064:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.570195:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29204 count 1 00010000:00010000:2.0:1785220581.570207:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570307:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005542ab08/0xf62499ab927855d9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a521 expref: 390 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570321:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005542ab08/0xf62499ab927855d9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a521 expref: 390 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570331:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.570333:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.570336:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570343:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570358:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.570378:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+389:2406:x1871938519924096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1642us (3302us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.570384:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100132 00000100:00100000:1.0:1785220581.570430:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284209 00000100:00100000:1.0:1785220581.570439:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1137:3653:x1871938510632704:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.570452:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29208 count 1 00010000:00010000:1.0:1785220581.570463:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570475:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001373951a/0xf62499ab92788995 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be6c expref: 344 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570481:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7218:0x0].0x0 since=28s v=(0 0) 00010000:00010000:1.0:1785220581.570484:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7218:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.570487:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570507:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001373951a/0xf62499ab92788995 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be6c expref: 344 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570516:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001373951a/0xf62499ab92788995 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be6c expref: 344 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.570526:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510633472:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.570558:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9ae74c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.570576:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284210 00000100:00100000:1.0:1785220581.570585:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1064:3655:x1871938510632576:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.570596:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29152 count 1 00010000:00010000:1.0:1785220581.570608:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570619:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000029df55e5/0xf62499ab9278583a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a640 expref: 388 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570625:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e0:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.570628:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.570632:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570649:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000029df55e5/0xf62499ab9278583a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a640 expref: 388 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570658:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000029df55e5/0xf62499ab9278583a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a640 expref: 388 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.570666:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510633600:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.570688:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777b40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.570717:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570726:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570734:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570741:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.570753:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220581.570913:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078d1a488/0xf62499ab9278859e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc5f expref: 344 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.570921:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7214:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.570924:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7214:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.570928:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.570948:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078d1a488/0xf62499ab9278859e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc5f expref: 344 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.570957:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078d1a488/0xf62499ab9278859e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc5f expref: 344 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.570966:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510633728:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.571075:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.571108:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571118:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.571122:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571132:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571146:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571154:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a49f0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571173:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.571178:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571185:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.571187:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571195:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571206:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571288:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a96fe265/0xf62499ab927a4a0c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571308:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.571312:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571318:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.571321:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571329:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571339:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571348:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a49db lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.571361:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.571363:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.571365:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.571485:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.571795:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1065:3652:x1871938510629632:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20740us (21060us total) trans 4295069917 rc 0/0 00000100:00100000:1.0:1785220581.571807:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284195 00010000:00010000:1.0:1785220581.571913:0:14227:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062c09318/0xf62499ab927854cf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a495 expref: 388 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.571926:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d977f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510631168:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.571945:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.571954:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.571960:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.571969:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.571976:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.571992:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.572004:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.572009:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572016:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.572019:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572026:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572038:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572045:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a49fe lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572061:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.572063:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.572064:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.572074:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572082:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572110:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572118:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.572121:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1065:3652:x1871938510630784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11548us (11700us total) trans 4295069918 rc 0/0 00000100:00100000:1.0:1785220581.572127:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284199 00010000:00010000:3.0:1785220581.572135:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.572167:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000732e6d4a/0xf62499ab927856c7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a591 expref: 387 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.572178:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfa5a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510631424:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.572189:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572197:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572204:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.572209:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1065:3654:x1871938510630912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11632us (11666us total) trans 4295069919 rc 0/0 00000100:00100000:3.0:1785220581.572216:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284200 00010000:00010000:3.0:1785220581.572233:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.572239:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572245:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.572248:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572255:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572265:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572272:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a49f7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.572288:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.572290:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.572292:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.572296:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572303:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572321:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.572391:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfa5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1065:3654:x1871938510631040:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10688us (10780us total) trans 4295069920 rc 0/0 00000100:00100000:1.0:1785220581.572397:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284201 00010000:00010000:1.0:1785220581.572421:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.572426:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572439:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.572442:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572449:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572460:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572466:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4a05 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.572481:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.572482:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.572484:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.572514:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d985e8a80/2, rc 0, transno 4295069916 00000100:00100000:1.0:1785220581.572526:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d985e8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510631552:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.572535:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb2680/2, rc 0, transno 4295069918 00000100:00100000:1.0:1785220581.572539:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510630784:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.572548:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb0000/2, rc 0, transno 4295069915 00000100:00100000:1.0:1785220581.572551:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510630528:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.572560:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb3100/2, rc 0, transno 4295069917 00000100:00100000:1.0:1785220581.572564:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510629632:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.572577:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04700/2, rc 0, transno 4295070508 00000100:00100000:1.0:1785220581.572585:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510629248:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.572597:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.572663:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.572673:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510633856:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.572706:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.572923:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510633856 00000100:00100000:2.0:1785220581.572930:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510633856 00000100:00100000:2.0:1785220581.572964:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284211 00000100:00100000:2.0:1785220581.572972:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1065:3652:x1871938510633856:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.572993:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29153 count 1 00010000:00010000:2.0:1785220581.573010:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.573025:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000568604e4/0xf62499ab927859d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a6d3 expref: 386 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.573033:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e1:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.573037:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.573040:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.573068:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000568604e4/0xf62499ab927859d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a6d3 expref: 386 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.573080:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000568604e4/0xf62499ab927859d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a6d3 expref: 386 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.573115:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510634368:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.573161:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.573175:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.573312:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510634240:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.573336:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.573352:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510634240 00000100:00100000:3.0:1785220581.573356:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510634240 00000100:00100000:3.0:1785220581.573379:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284212 00000100:00100000:3.0:1785220581.573385:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1066:3652:x1871938510634240:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.573396:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29156 count 1 00010000:00010000:3.0:1785220581.573409:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573415:0:13866:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019c0d88e/0xf62499ab927852de lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a3a0 expref: 386 pid: 14251 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573420:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004cac8f1c/0xf62499ab92785d26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a838 expref: 386 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573427:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e4:0x0].0x0 since=30s v=(0 0) 00000100:00100000:0.0:1785220581.573428:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510630400:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.573430:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.573433:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573444:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573450:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004cac8f1c/0xf62499ab92785d26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a838 expref: 385 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573453:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573460:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004cac8f1c/0xf62499ab92785d26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a838 expref: 385 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573461:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573470:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.573471:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510634624:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.573478:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.573493:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.573493:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.573510:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510634496:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.573533:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.573561:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510634496 00000100:00100000:3.0:1785220581.573565:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510634496 00000100:00100000:0.0:1785220581.573584:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1067:3655:x1871938510629760:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21750us (21971us total) trans 4295069921 rc 0/0 00000100:00100000:3.0:1785220581.573586:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284213 00000100:00100000:0.0:1785220581.573591:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284197 00000100:00100000:3.0:1785220581.573592:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1138:3653:x1871938510634496:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.573602:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29209 count 1 00010000:00010000:3.0:1785220581.573616:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573628:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000792eaf0c/0xf62499ab92788a59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bece expref: 344 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573632:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.573634:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7219:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.573637:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7219:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.573638:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573641:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573646:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.573649:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573656:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573659:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000792eaf0c/0xf62499ab92788a59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bece expref: 344 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.573669:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000792eaf0c/0xf62499ab92788a59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bece expref: 344 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573671:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.573678:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a49e9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.573680:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510634752:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.573695:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.573696:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.573698:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.573702:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.573707:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.574205:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510633344:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.574222:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.574354:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510633216:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.574368:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.574390:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d977f2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510634112:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.574417:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.574453:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb3800/2, rc 0, transno 4295069921 00000100:00100000:2.0:1785220581.574458:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510629760:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.574467:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.574472:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfa5a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510633984:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.574492:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.574513:0:14159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510634112 00000100:00100000:2.0:1785220581.574517:0:14159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510634112 00000100:00100000:2.0:1785220581.574541:0:14159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510633984 00000100:00100000:2.0:1785220581.574544:0:14159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510633984 00000100:00100000:2.0:1785220581.574571:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284214 00000100:00100000:2.0:1785220581.574578:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1068:3655:x1871938510634112:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.574595:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29155 count 1 00010000:00010000:2.0:1785220581.574610:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574622:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078127e83/0xf62499ab92785bdd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7c8 expref: 385 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574630:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e3:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.574633:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.574636:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574659:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078127e83/0xf62499ab92785bdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7c8 expref: 385 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574669:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078127e83/0xf62499ab92785bdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7c8 expref: 385 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.574680:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510634880:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.574717:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.574734:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284215 00000100:00100000:2.0:1785220581.574741:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1068:3655:x1871938510633984:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.574752:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29154 count 1 00010000:00010000:2.0:1785220581.574777:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574791:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe4dfc29/0xf62499ab92785ae1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a751 expref: 385 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574798:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e2:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.574801:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.574806:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574824:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fe4dfc29/0xf62499ab92785ae1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a751 expref: 385 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.574845:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fe4dfc29/0xf62499ab92785ae1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a751 expref: 385 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.574857:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0001_UUID:14192:1871938510635008:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.574880:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.574911:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99064000/2, rc 0, transno 4295069920 00000100:00100000:2.0:1785220581.574917:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510631040:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.574927:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb1180/2, rc 0, transno 4295069919 00000100:00100000:2.0:1785220581.574930:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510630912:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.575357:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510632960:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.575374:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.575449:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.575459:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510635392:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.575487:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.575514:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510635264:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.575536:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.575550:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.575555:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510635136:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.575596:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.575618:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510635392 00000100:00100000:2.0:1785220581.575622:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510635392 00000100:00100000:2.0:1785220581.575657:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510635264 00000100:00100000:2.0:1785220581.575660:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510635264 00000100:00100000:2.0:1785220581.575678:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510635136 00000100:00100000:2.0:1785220581.575680:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510635136 00000100:00100000:2.0:1785220581.575701:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284216 00000100:00100000:2.0:1785220581.575708:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1071:3653:x1871938510635392:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.575723:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29159 count 1 00010000:00010000:2.0:1785220581.575739:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.575751:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000aff6fc2f/0xf62499ab92785f8e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a957 expref: 385 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.575758:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e7:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.575773:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.575777:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.575797:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000aff6fc2f/0xf62499ab92785f8e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a957 expref: 385 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.575808:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000aff6fc2f/0xf62499ab92785f8e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a957 expref: 385 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.575819:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510635520:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.576077:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.576117:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284217 00000100:00100000:2.0:1785220581.576126:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebd8f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1071:3655:x1871938510635264:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.576202:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29158 count 1 00010000:00010000:2.0:1785220581.576287:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576301:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005522dcd8/0xf62499ab92785efb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a90a expref: 385 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576308:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e6:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.576312:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.576316:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576336:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005522dcd8/0xf62499ab92785efb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a90a expref: 385 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576352:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005522dcd8/0xf62499ab92785efb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a90a expref: 385 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.576362:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510635648:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.576391:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.576409:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284218 00000100:00100000:2.0:1785220581.576417:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1071:3655:x1871938510635136:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.576430:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29157 count 1 00010000:00010000:2.0:1785220581.576441:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576453:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ee5d258c/0xf62499ab92785e68 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8c4 expref: 385 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576460:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e5:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.576463:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.576467:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576485:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ee5d258c/0xf62499ab92785e68 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8c4 expref: 385 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.576495:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ee5d258c/0xf62499ab92785e68 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8c4 expref: 385 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.576512:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510635776:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.576535:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.577347:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510633088:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.577377:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.577406:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510632832:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.577418:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.577516:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.577545:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.577572:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.577587:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.577607:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.577671:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510633728:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.577685:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.577729:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510634368:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.577742:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.577850:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510634880:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.577864:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.578126:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924480 00000100:00100000:0.0:1785220581.578143:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924480 00010000:00010000:0.0:1785220581.578174:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000971d3fc2/0xf62499ab927886cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bceb expref: 345 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.578200:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924736 00000100:00100000:0.0:1785220581.578203:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924736 00010000:00010000:0.0:1785220581.578332:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000797cd97f/0xf62499ab927888fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bdf5 expref: 346 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.578354:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924992 00000100:00100000:0.0:1785220581.578356:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924992 00010000:00010000:0.0:1785220581.578369:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000082297f5b/0xf62499ab92788440 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb78 expref: 347 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.578385:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924608 00000100:00100000:0.0:1785220581.578388:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924608 00010000:00010000:0.0:1785220581.578398:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000536dec88/0xf62499ab92788796 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd5b expref: 348 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.578413:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519924864 00000100:00100000:0.0:1785220581.578415:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519924864 00010000:00010000:0.0:1785220581.578427:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ec85ab24/0xf62499ab92788504 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bbd3 expref: 349 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.578444:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100136 00000100:00100000:0.0:1785220581.578453:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+349:2407:x1871938519924480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.578465:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.578472:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000971d3fc2/0xf62499ab927886cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bceb expref: 349 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578483:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000971d3fc2/0xf62499ab927886cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bceb expref: 349 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578511:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000971d3fc2/0xf62499ab927886cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bceb expref: 349 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578528:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7215:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.578532:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7215:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.578538:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578548:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578571:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.578613:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+348:2407:x1871938519924480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 161us (1097us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.578621:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100136 00000100:00100000:0.0:1785220581.578638:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100137 00000100:00100000:0.0:1785220581.578644:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+347:2407:x1871938519924736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.578652:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.578657:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000797cd97f/0xf62499ab927888fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bdf5 expref: 347 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578666:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000797cd97f/0xf62499ab927888fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bdf5 expref: 347 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578684:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000797cd97f/0xf62499ab927888fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bdf5 expref: 347 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578694:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7217:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.578697:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7217:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.578701:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578708:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.578723:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.578743:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510634752:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.578771:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.578801:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.578808:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100138 00010000:00010000:2.0:1785220581.578810:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.578815:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+346:2407:x1871938519924992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.578820:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.578823:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.578827:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000082297f5b/0xf62499ab92788440 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb78 expref: 346 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.578828:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.578837:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000082297f5b/0xf62499ab92788440 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb78 expref: 346 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.578850:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082297f5b/0xf62499ab92788440 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb78 expref: 346 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.578854:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.578860:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7212:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.578863:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7212:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.578867:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.578877:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.578892:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.579052:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100139 00000100:00100000:1.0:1785220581.579062:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+345:2407:x1871938519924608:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.579069:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.579074:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000536dec88/0xf62499ab92788796 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd5b expref: 345 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.579096:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000536dec88/0xf62499ab92788796 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd5b expref: 345 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.579111:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536dec88/0xf62499ab92788796 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd5b expref: 345 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.579121:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7216:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.579124:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7216:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.579473:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.579485:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.579509:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.579575:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+344:2407:x1871938519924736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 929us (2027us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.579584:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100137 00000100:00100000:0.0:1785220581.579607:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100140 00000100:00100000:0.0:1785220581.579615:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+343:2407:x1871938519924864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.579624:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.579630:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ec85ab24/0xf62499ab92788504 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bbd3 expref: 343 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579641:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ec85ab24/0xf62499ab92788504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bbd3 expref: 343 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579660:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec85ab24/0xf62499ab92788504 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bbd3 expref: 343 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579671:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7213:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.579675:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7213:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.579679:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579687:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579708:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.579726:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+342:2407:x1871938519924864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (2118us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.579733:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100140 00010000:00010000:0.0:1785220581.579755:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579775:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579784:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579791:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579806:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.579932:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1140:3653:x1871938510632320:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12021us (13988us total) trans 4295070511 rc 0/0 00000100:00100000:0.0:1785220581.579939:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284205 00010000:00010000:0.0:1785220581.579964:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579973:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579981:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.579988:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580002:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.580071:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+1140:3653:x1871938510631808:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12599us (14377us total) trans 4295070512 rc 0/0 00000100:00100000:0.0:1785220581.580078:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284203 00010000:00010000:0.0:1785220581.580111:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.580118:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580125:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.580291:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580303:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580315:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580322:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4a1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.580335:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+341:2407:x1871938519924992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1519us (2761us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.580341:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:3.0:1785220581.580342:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100138 00010000:00010000:0.0:1785220581.580345:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580351:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.580355:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580360:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.580362:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580364:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580371:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.580372:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580375:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580379:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4a21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580383:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580393:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580393:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.580395:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.580397:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.580401:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a4a13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580413:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.580415:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.580417:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.580443:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510634624:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.580460:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580482:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.580486:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.580490:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580497:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580505:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.580515:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.580524:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.580530:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.580534:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580540:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.580543:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580551:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580560:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580568:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a4a2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580582:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.580583:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.580585:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.580605:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1140:3654:x1871938510632448:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11005us (14595us total) trans 4295070513 rc 0/0 00000100:00100000:3.0:1785220581.580612:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284207 00000100:00100000:0.0:1785220581.580612:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.580628:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519925248 00000100:00100000:0.0:1785220581.580633:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519925248 00010000:00010000:0.0:1785220581.580653:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000029df55e5/0xf62499ab9278583a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a640 expref: 386 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.580675:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519925504 00000100:00100000:0.0:1785220581.580678:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519925504 00010000:00010000:0.0:1785220581.580691:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000568604e4/0xf62499ab927859d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a6d3 expref: 387 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.580705:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519925760 00000100:00100000:0.0:1785220581.580708:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519925760 00010000:00010000:0.0:1785220581.580721:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000792eaf0c/0xf62499ab92788a59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bece expref: 341 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.580751:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100141 00000100:00100000:0.0:1785220581.580775:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e12450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+387:2407:x1871938519925248:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.580784:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100142 00010000:00010000:0.0:1785220581.580787:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220581.580791:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+387:2407:x1871938519925504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.580792:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000029df55e5/0xf62499ab9278583a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a640 expref: 387 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580798:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.580802:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000568604e4/0xf62499ab927859d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a6d3 expref: 387 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580803:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000029df55e5/0xf62499ab9278583a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a640 expref: 387 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580813:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000568604e4/0xf62499ab927859d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a6d3 expref: 387 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580820:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.580824:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000568604e4/0xf62499ab927859d0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a6d3 expref: 387 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580824:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.580829:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580835:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.580838:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.580842:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.580850:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580850:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580865:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.580868:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.580890:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e12450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+386:2407:x1871938519925248:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 131us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.580897:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100141 00000100:00100000:0.0:1785220581.580912:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100143 00000100:00100000:0.0:1785220581.580919:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+341:2407:x1871938519925760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.580926:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.580930:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000792eaf0c/0xf62499ab92788a59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bece expref: 341 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580940:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000792eaf0c/0xf62499ab92788a59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bece expref: 341 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580952:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000792eaf0c/0xf62499ab92788a59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bece expref: 341 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580963:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7219:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.580966:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7219:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.580970:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580982:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.580999:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.582822:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+385:2407:x1871938519925504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2022us (2295us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.582827:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+340:2407:x1871938519925760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1908us (2213us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.582838:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100143 00010000:00010000:0.0:1785220581.582880:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.582886:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100142 00010000:00010000:0.0:1785220581.582892:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.582902:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.582910:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.582929:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.583117:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1140:3653:x1871938510634496:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9524us (9581us total) trans 4295070514 rc 0/0 00000100:00100000:0.0:1785220581.583125:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284213 00010000:00010000:0.0:1785220581.583147:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.583153:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583161:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.583166:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583174:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583192:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583200:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4a59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583324:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.583327:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.583328:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.583386:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.583394:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.583397:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583406:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583421:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.583424:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.583428:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583441:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.583442:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926016 00000100:00100000:0.0:1785220581.583448:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926016 00010000:00010000:0.0:1785220581.583473:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fe4dfc29/0xf62499ab92785ae1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a751 expref: 385 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.583501:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519925120 00000100:00100000:0.0:1785220581.583504:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519925120 00010000:00010000:0.0:1785220581.583518:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001373951a/0xf62499ab92788995 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be6c expref: 340 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.583538:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100144 00000100:00100000:0.0:1785220581.583546:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+385:2407:x1871938519926016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.583551:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1140:3654:x1871938510631936:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14380us (17561us total) trans 4295070515 rc 0/0 00000100:00100000:3.0:1785220581.583557:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284206 00010000:00010000:0.0:1785220581.583558:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.583563:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fe4dfc29/0xf62499ab92785ae1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a751 expref: 385 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583573:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fe4dfc29/0xf62499ab92785ae1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a751 expref: 385 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.583583:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100145 00000100:00100000:3.0:1785220581.583590:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+340:2407:x1871938519925120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.583590:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.583595:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.583598:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.583600:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583603:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001373951a/0xf62499ab92788995 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be6c expref: 340 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583610:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583612:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001373951a/0xf62499ab92788995 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be6c expref: 340 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583625:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7218:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.583628:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7218:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.583628:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.583632:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583645:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.583645:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+385:2407:x1871938519926016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.583651:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100144 00010000:00010000:3.0:1785220581.583657:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.583671:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+340:2407:x1871938519925120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.583678:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100145 00010000:00010000:3.0:1785220581.583694:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1785220581.583697:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.583699:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583706:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fec00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.583709:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583717:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.583723:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.583729:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.583735:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519925376 00010000:00010000:3.0:1785220581.583737:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002900884/0xf62499ab927a4a28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.583739:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519925376 00010000:00010000:3.0:1785220581.583752:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.583753:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078d1a488/0xf62499ab9278859e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc5f expref: 340 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.583754:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.583756:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.583783:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519925632 00000100:00100000:0.0:1785220581.583786:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519925632 00010000:00010000:0.0:1785220581.583798:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004cac8f1c/0xf62499ab92785d26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a838 expref: 385 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.583804:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725880/2, rc 0, transno 4295070514 00000100:00100000:0.0:1785220581.583813:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100146 00000100:00100000:3.0:1785220581.583816:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510634496:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.583819:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+340:2407:x1871938519925376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.583826:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.583830:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078d1a488/0xf62499ab9278859e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc5f expref: 340 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.583831:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04000/2, rc 0, transno 4295070511 00010000:00010000:0.0:1785220581.583839:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078d1a488/0xf62499ab9278859e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc5f expref: 340 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583851:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078d1a488/0xf62499ab9278859e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc5f expref: 340 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583861:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7214:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.583864:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7214:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.583868:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583876:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583895:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.583914:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+339:2407:x1871938519925376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (216us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.583920:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100146 00000100:00100000:0.0:1785220581.583933:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100147 00000100:00100000:0.0:1785220581.583939:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+385:2407:x1871938519925632:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.583946:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.583951:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004cac8f1c/0xf62499ab92785d26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a838 expref: 385 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583960:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004cac8f1c/0xf62499ab92785d26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a838 expref: 385 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583972:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cac8f1c/0xf62499ab92785d26 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a838 expref: 385 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583982:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.583985:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.583989:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.583995:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.584015:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.584031:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+384:2407:x1871938519925632:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.584037:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100147 00010000:00010000:0.0:1785220581.584055:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.584064:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.584065:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510632320:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.584073:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.584080:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.584082:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbea00/2, rc 0, transno 4295070512 00010000:00010000:0.0:1785220581.584111:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.584112:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510631808:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.584161:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.584217:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.584235:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.584247:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519925888 00000100:00100000:3.0:1785220581.584250:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519925888 00010000:00010000:3.0:1785220581.584265:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078127e83/0xf62499ab92785bdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7c8 expref: 384 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.584283:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926144 00000100:00100000:3.0:1785220581.584286:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926144 00000100:00100000:0.0:1785220581.584288:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1072:3652:x1871938510634240:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10825us (10872us total) trans 4295069922 rc 0/0 00000100:00100000:0.0:1785220581.584297:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284212 00010000:00010000:3.0:1785220581.584298:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000aff6fc2f/0xf62499ab92785f8e lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a957 expref: 385 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.584311:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926400 00000100:00100000:3.0:1785220581.584313:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926400 00000100:00100000:0.0:1785220581.584316:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100148 00000100:00100000:0.0:1785220581.584321:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90703c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+386:2407:x1871938519925888:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.584324:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ee5d258c/0xf62499ab92785e68 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8c4 expref: 386 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.584329:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.584333:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078127e83/0xf62499ab92785bdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7c8 expref: 386 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.584336:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100149 00000100:00100000:3.0:1785220581.584342:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+386:2406:x1871938519926144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.584344:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078127e83/0xf62499ab92785bdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7c8 expref: 386 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584348:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.584352:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aff6fc2f/0xf62499ab92785f8e lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a957 expref: 386 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.584356:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078127e83/0xf62499ab92785bdd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7c8 expref: 386 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584361:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aff6fc2f/0xf62499ab92785f8e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a957 expref: 386 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.584366:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.584368:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.584372:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.584372:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584374:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.584378:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.584380:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584386:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584395:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.584398:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.584413:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+385:2406:x1871938519926144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 70us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.584419:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100149 00000100:00100000:3.0:1785220581.584432:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100150 00000100:00100000:2.0:1785220581.584478:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e05500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1139:3653:x1871938510632192:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16777us (18668us total) trans 4295070510 rc 0/0 00000100:00100000:2.0:1785220581.584486:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284204 00000100:00100000:3.0:1785220581.584490:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+384:2406:x1871938519926400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.584497:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.584501:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ee5d258c/0xf62499ab92785e68 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8c4 expref: 384 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584516:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ee5d258c/0xf62499ab92785e68 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8c4 expref: 384 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584520:0:14192:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe4dfc29/0xf62499ab92785ae1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a751 expref: 384 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584528:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e5:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220581.584530:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0001_UUID:14192:1871938510635008:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.584531:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.584535:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584541:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584546:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584552:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.584554:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584561:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.584567:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2406:x1871938519926400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (333us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.584570:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.584573:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100150 00010000:00010000:2.0:1785220581.584578:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584592:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1785220581.584599:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbdf80/2, rc 0, transno 4295070510 00000100:00100000:3.0:1785220581.584603:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510632192:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.584634:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.584647:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926272 00000100:00100000:3.0:1785220581.584650:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926272 00010000:00010000:3.0:1785220581.584664:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005522dcd8/0xf62499ab92785efb lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a90a expref: 383 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.584677:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1072:3655:x1871938510633984:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9937us (10184us total) trans 4295069923 rc 0/0 00000100:00100000:3.0:1785220581.584682:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100151 00000100:00100000:2.0:1785220581.584683:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284215 00000100:00100000:3.0:1785220581.584688:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2406:x1871938519926272:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.584696:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.584700:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005522dcd8/0xf62499ab92785efb lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a90a expref: 383 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584704:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584709:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005522dcd8/0xf62499ab92785efb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a90a expref: 383 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584713:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584721:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584722:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.584725:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.584729:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584730:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584738:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584741:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:3.0:1785220581.584748:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.584780:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2406:x1871938519926272:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (145us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.584786:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100151 00010000:00010000:3.0:1785220581.584802:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.584807:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584815:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.584819:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.584822:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1072:3652:x1871938510633856:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11852us (12116us total) trans 4295069924 rc 0/0 00000100:00100000:2.0:1785220581.584829:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284211 00010000:00010000:2.0:1785220581.584857:0:14198:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005522dcd8/0xf62499ab92785efb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a90a expref: 382 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.584873:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510635648:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.584885:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584894:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584900:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584908:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584914:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.584928:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:3.0:1785220581.584967:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584980:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.584987:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4a52 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.584994:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebd8f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1072:3655:x1871938510635264:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8870us (9457us total) trans 4295069925 rc 0/0 00000100:00100000:2.0:1785220581.584999:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284217 00010000:00010000:3.0:1785220581.585005:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.585008:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585015:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ffc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.585018:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585027:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585027:0:13837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee5d258c/0xf62499ab92785e68 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8c4 expref: 381 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.585037:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510635776:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.585038:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585045:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005275d713/0xf62499ab927a4a67 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585048:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585057:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585064:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.585064:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585067:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585072:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585073:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fe200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.585076:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585080:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585109:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.585176:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1072:3655:x1871938510635136:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8760us (9578us total) trans 4295069926 rc 0/0 00000100:00100000:2.0:1785220581.585183:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284218 00010000:00010000:2.0:1785220581.585199:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.585205:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585213:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585227:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585234:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000081ef1dad/0xf62499ab927a4a4b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585249:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.585253:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585259:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.585262:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585275:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585284:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585292:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4a7c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585304:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.585306:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.585307:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.585327:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+338:2407:x1871938519924608:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 6266us (7737us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.585335:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100139 00010000:00010000:1.0:1785220581.585398:0:14149:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001373951a/0xf62499ab92788995 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be6c expref: 337 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.585409:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510633472:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.585423:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585431:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585437:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585445:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585450:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585475:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.585572:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1139:3653:x1871938510632704:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15135us (19061us total) trans 4295070516 rc 0/0 00000100:00100000:1.0:1785220581.585579:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284209 00010000:00010000:1.0:1785220581.585608:0:13836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000029df55e5/0xf62499ab9278583a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a640 expref: 380 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.585620:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510633600:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.585633:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585640:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585647:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585654:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585662:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.585670:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.585725:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1072:3655:x1871938510632576:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15142us (19177us total) trans 4295069927 rc 0/0 00000100:00100000:1.0:1785220581.585729:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284210 00000004:00080000:1.0:1785220581.585801:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99064e00/2, rc 0, transno 4295069922 00010000:00010000:2.0:1785220581.585804:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.585808:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.585812:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510634240:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.585821:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.585821:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ec380/2, rc 0, transno 4295069924 00000100:00100000:1.0:1785220581.585825:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510633856:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.585879:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.585884:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585892:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.585895:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585902:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585907:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585914:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585914:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4a75 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585926:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a4a3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585932:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.585934:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.585940:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.585941:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.585941:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.585943:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.585944:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585952:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585960:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.585961:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91479880/2, rc 0, transno 4295070516 00000100:00100000:3.0:1785220581.585966:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91479880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510632704:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.585968:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4a44 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.585986:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.585988:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.585990:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.586007:0:14210:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aff6fc2f/0xf62499ab92785f8e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a957 expref: 379 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.586014:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510635520:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.586021:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586027:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586031:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586035:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586039:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586051:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.586070:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.586075:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586081:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960ea1046a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.586136:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586144:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586155:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586162:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008aee5a0a/0xf62499ab927a4a6e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586179:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.586181:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.586183:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.586355:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1072:3653:x1871938510635392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10647us (10866us total) trans 4295069928 rc 0/0 00000100:00100000:2.0:1785220581.586362:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284216 00010000:00010000:2.0:1785220581.586384:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586392:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586399:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586406:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586418:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.586490:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1072:3655:x1871938510634112:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11911us (12071us total) trans 4295069929 rc 0/0 00000100:00100000:2.0:1785220581.586497:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284214 00010000:00010000:2.0:1785220581.586512:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.586515:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586522:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902fea00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.586525:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586533:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586544:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586550:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a11d065a/0xf62499ab927a4a60 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586564:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.586566:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.586567:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.586573:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.586584:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586592:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586600:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586606:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.586617:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.586693:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1139:3654:x1871938510632064:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16514us (20660us total) trans 4295070517 rc 0/0 00000100:00100000:2.0:1785220581.586702:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284208 00010000:00010000:3.0:1785220581.586796:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.586800:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586807:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d902ff200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.586810:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586816:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.586826:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.586898:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636160:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.586898:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510636288:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.586927:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.586949:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f9e153b/0xf62499ab927a4a36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.586963:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510636160 00000100:00100000:2.0:1785220581.586967:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510636160 00010000:00010000:3.0:1785220581.586973:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.586974:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.586976:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.586986:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510635904:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.586999:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284219 00000100:00100000:2.0:1785220581.587006:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1139:3655:x1871938510636160:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.587011:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:2.0:1785220581.587030:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29210 count 1 00000100:00100000:3.0:1785220581.587031:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510636416:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.587047:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.587049:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.587064:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000287ba917/0xf62499ab92788b39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf30 expref: 336 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.587070:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa2a00/2, rc 0, transno 4295069928 00010000:00010000:2.0:1785220581.587072:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x721a:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.587076:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721a:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.587077:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510635392:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.587081:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.587123:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.588068:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510635904 00000100:00100000:0.0:1785220581.588070:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90703c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+378:2407:x1871938519925888:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 3745us (3905us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.588074:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510635904 00000100:00100000:0.0:1785220581.588080:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100148 00010000:00010000:0.0:1785220581.588145:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.588223:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000287ba917/0xf62499ab92788b39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf30 expref: 336 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.588237:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000287ba917/0xf62499ab92788b39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf30 expref: 336 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.588257:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510636800:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.588300:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.588320:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510636416 00000100:00100000:3.0:1785220581.588328:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510636416 00000100:00100000:1.0:1785220581.588366:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510636288 00000100:00100000:1.0:1785220581.588369:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510636288 00000100:00100000:1.0:1785220581.588407:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284220 00000100:00100000:1.0:1785220581.588415:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1072:3653:x1871938510635904:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.588436:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29160 count 1 00010000:00010000:1.0:1785220581.588453:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.588467:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002d650261/0xf62499ab927860a6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9c7 expref: 377 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.588474:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e8:0x0].0x0 since=30s v=(0 0) 00010000:00010000:1.0:1785220581.588477:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.588481:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.588499:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002d650261/0xf62499ab927860a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9c7 expref: 377 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.588509:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002d650261/0xf62499ab927860a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9c7 expref: 377 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.588519:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510636928:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.588543:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777b40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.588595:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284221 00000100:00100000:0.0:1785220581.588603:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1141:3653:x1871938510636416:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.588615:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29212 count 1 00010000:00010000:0.0:1785220581.588629:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.588646:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000070fe3c19/0xf62499ab92788d4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c041 expref: 336 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.588654:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x721c:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.588657:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.588661:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.588680:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000070fe3c19/0xf62499ab92788d4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c041 expref: 336 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.588690:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000070fe3c19/0xf62499ab92788d4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c041 expref: 336 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.588701:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510637056:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.588723:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.588818:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284222 00000100:00100000:2.0:1785220581.588827:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1141:3652:x1871938510636288:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.588839:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29211 count 1 00010000:00010000:2.0:1785220581.588851:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.588862:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000515aacde/0xf62499ab92788c51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bfca expref: 336 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.588869:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x721b:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.588872:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.588875:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.588894:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000515aacde/0xf62499ab92788c51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bfca expref: 336 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.588904:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000515aacde/0xf62499ab92788c51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bfca expref: 336 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.588915:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfa5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510637184:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.588938:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.589003:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636672:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.589009:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.589018:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510637312:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.589035:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.589070:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510636672 00000100:00100000:0.0:1785220581.589074:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510636672 00000004:00080000:2.0:1785220581.589173:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa0000/2, rc 0, transno 4295069926 00000100:00100000:2.0:1785220581.589184:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510635136:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.589199:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa0e00/2, rc 0, transno 4295069925 00000100:00100000:2.0:1785220581.589202:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510635264:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.589210:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfa5a300/2, rc 0, transno 4295069923 00000100:00100000:2.0:1785220581.589213:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfa5a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510633984:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.589222:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d977f2d80/2, rc 0, transno 4295069929 00000100:00100000:2.0:1785220581.589225:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d977f2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510634112:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.589232:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991b480/2, rc 0, transno 4295069927 00000100:00100000:2.0:1785220581.589236:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510632576:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.589244:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1785220581.589248:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636032:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.589270:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.589291:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510636032 00000100:00100000:2.0:1785220581.589294:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510636032 00000100:00100000:2.0:1785220581.589317:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284223 00000100:00100000:2.0:1785220581.589323:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1073:3655:x1871938510636032:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.589336:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29161 count 1 00000100:00100000:3.0:1785220581.589347:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.589349:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589360:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001d5fa4c4/0xf62499ab92786178 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa30 expref: 377 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589367:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71e9:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.589370:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.589374:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589392:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001d5fa4c4/0xf62499ab92786178 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa30 expref: 377 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589402:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001d5fa4c4/0xf62499ab92786178 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa30 expref: 377 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.589412:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510637440:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.589440:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.589532:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510637312 00000100:00100000:0.0:1785220581.589536:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510637312 00000100:00100000:0.0:1785220581.589561:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284224 00000100:00100000:0.0:1785220581.589568:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1142:3655:x1871938510636672:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.589579:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29214 count 1 00010000:00010000:0.0:1785220581.589591:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.589608:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005433698e/0xf62499ab92788f8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c14b expref: 336 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.589615:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x721e:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.589617:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.589621:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.589639:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005433698e/0xf62499ab92788f8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c14b expref: 336 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.589648:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005433698e/0xf62499ab92788f8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c14b expref: 336 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.589657:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510637568:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.589681:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d87b9b300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.589686:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284225 00000100:00100000:2.0:1785220581.589694:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1074:3653:x1871938510637312:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.589706:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29162 count 1 00010000:00010000:2.0:1785220581.589718:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589730:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ff8d1cdd/0xf62499ab9278624a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aaa7 expref: 377 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589737:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ea:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.589740:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.589745:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589771:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ff8d1cdd/0xf62499ab9278624a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aaa7 expref: 377 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.589781:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ff8d1cdd/0xf62499ab9278624a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aaa7 expref: 377 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.589791:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510637696:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.589811:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.589864:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa61500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510637824:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.589888:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636544:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.589889:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.589911:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.589918:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510638336:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.589927:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510638080:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.589938:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.589950:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510637952:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.589962:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510637824 00000100:00100000:3.0:1785220581.589964:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.589965:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510637824 00000100:00100000:3.0:1785220581.589969:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510638208:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.589985:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510636544 00000100:00100000:3.0:1785220581.589988:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.589988:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510636544 00000100:00100000:2.0:1785220581.590000:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510638336 00000100:00100000:2.0:1785220581.590003:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510638336 00000100:00100000:2.0:1785220581.590029:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510638208 00000100:00100000:2.0:1785220581.590032:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510638208 00000100:00100000:0.0:1785220581.590040:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284226 00000100:00100000:0.0:1785220581.590047:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1077:3653:x1871938510637824:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.590053:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284227 00000100:00100000:2.0:1785220581.590058:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1143:3655:x1871938510636544:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.590059:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29163 count 1 00002000:00080000:2.0:1785220581.590067:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29213 count 1 00010000:00010000:0.0:1785220581.590071:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590077:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.590082:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009c104a5f/0xf62499ab9278638c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab48 expref: 377 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.590124:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71eb:0x0].0x0 since=30s v=(0 0) 00010000:00010000:0.0:1785220581.590127:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.590249:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8d68a/0xf62499ab92788eb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0e9 expref: 336 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590256:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x721d:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.590259:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.590263:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 4/0,1 mode: --/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590284:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ceb8d68a/0xf62499ab92788eb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0e9 expref: 336 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590294:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8d68a/0xf62499ab92788eb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0e9 expref: 336 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.590303:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284228 00000100:00100000:2.0:1785220581.590304:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510638464:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.590311:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1077:3653:x1871938510638336:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.590323:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29167 count 1 00000100:00100000:2.0:1785220581.590329:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.590335:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.590343:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284229 00010000:00010000:3.0:1785220581.590346:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ddf6b984/0xf62499ab927866e2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7acf3 expref: 377 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.590349:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1077:3653:x1871938510638208:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.590359:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ef:0x0].0x0 since=30s v=(0 0) 00002000:00080000:2.0:1785220581.590360:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29166 count 1 00010000:00010000:3.0:1785220581.590361:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.590365:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590369:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590379:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b1589c39/0xf62499ab92786592 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac4b expref: 377 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.590382:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ddf6b984/0xf62499ab927866e2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7acf3 expref: 377 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590386:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ee:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.590389:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.590392:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ddf6b984/0xf62499ab927866e2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7acf3 expref: 377 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590393:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.590403:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0001_UUID:14259:1871938510638592:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.590408:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b1589c39/0xf62499ab92786592 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac4b expref: 377 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590417:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b1589c39/0xf62499ab92786592 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac4b expref: 377 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.590420:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.590426:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510638720:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.590444:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.590451:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.590466:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.590471:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009c104a5f/0xf62499ab9278638c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab48 expref: 377 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.590480:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009c104a5f/0xf62499ab9278638c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab48 expref: 377 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.590490:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510638848:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.590507:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.590518:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.590520:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510638080 00000100:00100000:0.0:1785220581.590522:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510638080 00000100:00100000:2.0:1785220581.590537:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510637952 00000100:00100000:2.0:1785220581.590540:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510637952 00000100:00100000:0.0:1785220581.590545:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284230 00000100:00100000:0.0:1785220581.590550:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1079:3655:x1871938510638080:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.590560:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284231 00002000:00080000:0.0:1785220581.590561:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29165 count 1 00000100:00100000:2.0:1785220581.590564:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1079:3654:x1871938510637952:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.590571:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.590573:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29164 count 1 00010000:00010000:2.0:1785220581.590581:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.590583:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004d8cd115/0xf62499ab927864b9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abf0 expref: 377 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590592:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000071202272/0xf62499ab92786481 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abc6 expref: 377 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.590596:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ed:0x0].0x0 since=30s v=(0 0) 00010000:00010000:0.0:1785220581.590599:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.590603:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590605:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ec:0x0].0x0 since=30s v=(0 0) 00010000:00010000:2.0:1785220581.590607:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.590611:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.590620:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004d8cd115/0xf62499ab927864b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abf0 expref: 377 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590625:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000071202272/0xf62499ab92786481 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abc6 expref: 377 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.590630:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004d8cd115/0xf62499ab927864b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abf0 expref: 377 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.590634:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000071202272/0xf62499ab92786481 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abc6 expref: 377 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.590640:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510638976:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.590643:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510639104:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.590657:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.590686:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbdc00/2, rc 0, transno 4295070517 00000100:00100000:2.0:1785220581.590704:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510632064:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.590717:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbce00/2, rc 0, transno 4295070515 00000100:00100000:2.0:1785220581.590721:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510631936:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.590730:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e06a00/2, rc 0, transno 4295070513 00000100:00100000:2.0:1785220581.590734:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510632448:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.590852:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510639488:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.590886:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26a80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.591029:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.591064:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510639360:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.591282:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.591310:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510639488 00000100:00100000:2.0:1785220581.591315:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510639488 00000100:00100000:2.0:1785220581.591339:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510639360 00000100:00100000:2.0:1785220581.591342:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510639360 00000100:00100000:2.0:1785220581.591359:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284232 00000100:00100000:2.0:1785220581.591365:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1145:3652:x1871938510639488:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.591383:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29217 count 1 00010000:00010000:2.0:1785220581.591398:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.591413:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb1fed34/0xf62499ab92789232 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2a9 expref: 336 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.591420:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7221:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.591424:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7221:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.591428:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.591447:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fb1fed34/0xf62499ab92789232 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2a9 expref: 336 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.591457:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fb1fed34/0xf62499ab92789232 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2a9 expref: 336 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.591468:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510639616:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.591488:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284233 00000100:00100000:3.0:1785220581.591499:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1145:3655:x1871938510639360:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.591502:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00002000:00080000:3.0:1785220581.591510:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29216 count 1 00010000:00010000:3.0:1785220581.591522:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.591522:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.591527:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510639232:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.591535:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c0190141/0xf62499ab92789152 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c224 expref: 336 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.591541:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7220:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.591544:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7220:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.591549:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.591549:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.591566:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c0190141/0xf62499ab92789152 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c224 expref: 336 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.591584:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c0190141/0xf62499ab92789152 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c224 expref: 336 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.591593:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510639744:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.591616:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.597955:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510639232 00000100:00100000:0.0:1785220581.597969:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510639232 00000100:00100000:0.0:1785220581.598041:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284234 00000100:00100000:0.0:1785220581.598051:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1146:3655:x1871938510639232:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.598412:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.598432:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926528 00000100:00100000:3.0:1785220581.598436:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926528 00010000:00010000:3.0:1785220581.598461:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000287ba917/0xf62499ab92788b39 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf30 expref: 337 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.598491:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100152 00000100:00100000:3.0:1785220581.598498:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2407:x1871938519926528:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.598507:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.598512:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000287ba917/0xf62499ab92788b39 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf30 expref: 337 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.598523:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000287ba917/0xf62499ab92788b39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf30 expref: 337 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.598553:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.598557:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.598563:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.598573:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.598593:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.598637:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2407:x1871938519926528:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 139us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.598645:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100152 00000100:00100000:3.0:1785220581.598685:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.598782:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.598805:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.598898:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0001_UUID:14259:1871938510638592:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.598920:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.598988:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.599008:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.599026:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.599046:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.599065:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.599081:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.599129:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.599394:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfa5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510637184:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.599411:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.599437:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510637696:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.599449:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.599472:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510638720:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.599485:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.599508:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510638464:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.599519:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.599543:0:14166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000287ba917/0xf62499ab92788b39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf30 expref: 336 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.599555:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510636800:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.599566:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.599579:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.599587:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.599597:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.599605:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.599625:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.599813:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1147:3655:x1871938510636160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12805us (12883us total) trans 4295070518 rc 0/0 00000100:00100000:2.0:1785220581.599823:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284219 00000100:00100000:2.0:1785220581.599859:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510639104:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.599871:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.599905:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ed880/2, rc 0, transno 4295070518 00000100:00100000:2.0:1785220581.599918:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636160:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.599995:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510639872:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.600036:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.600211:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.600295:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600303:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.600307:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600315:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600331:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600338:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4a83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600357:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.600359:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.600361:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.600439:0:14250:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510639872 00000100:00100000:3.0:1785220581.600445:0:14250:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510639872 00000100:00100000:3.0:1785220581.600479:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284235 00000100:00100000:3.0:1785220581.600487:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1147:3653:x1871938510639872:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.600508:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29218 count 1 00010000:00010000:3.0:1785220581.600527:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600543:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005bf7ab59/0xf62499ab927892cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c30b expref: 335 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600551:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7222:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.600555:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7222:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.600559:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600586:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005bf7ab59/0xf62499ab927892cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c30b expref: 335 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.600599:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005bf7ab59/0xf62499ab927892cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c30b expref: 335 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.600611:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510640000:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.600717:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.600877:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510639744:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.600896:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.600968:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.600994:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00002000:00080000:0.0:1785220581.601060:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29215 count 1 00000100:00100000:0.0:1785220581.601119:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510637568:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.601142:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601168:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510637056:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.601185:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601280:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510638848:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.601296:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601323:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926784 00000100:00100000:0.0:1785220581.601328:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926784 00010000:00010000:0.0:1785220581.601350:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000070fe3c19/0xf62499ab92788d4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c041 expref: 336 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601374:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926656 00000100:00100000:0.0:1785220581.601378:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926656 00010000:00010000:0.0:1785220581.601392:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002d650261/0xf62499ab927860a6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9c7 expref: 378 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601414:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519926912 00000100:00100000:0.0:1785220581.601418:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519926912 00010000:00010000:0.0:1785220581.601430:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000515aacde/0xf62499ab92788c51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bfca expref: 337 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601449:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927040 00000100:00100000:0.0:1785220581.601452:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927040 00010000:00010000:0.0:1785220581.601463:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005433698e/0xf62499ab92788f8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c14b expref: 338 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601478:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927296 00000100:00100000:0.0:1785220581.601481:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927296 00010000:00010000:0.0:1785220581.601491:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8d68a/0xf62499ab92788eb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0e9 expref: 339 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601502:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927552 00000100:00100000:0.0:1785220581.601505:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927552 00010000:00010000:0.0:1785220581.601516:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b1589c39/0xf62499ab92786592 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac4b expref: 379 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601526:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927808 00000100:00100000:0.0:1785220581.601529:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927808 00010000:00010000:0.0:1785220581.601540:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000071202272/0xf62499ab92786481 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abc6 expref: 380 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601552:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927168 00000100:00100000:0.0:1785220581.601554:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927168 00010000:00010000:0.0:1785220581.601566:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ff8d1cdd/0xf62499ab9278624a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aaa7 expref: 381 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601577:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927424 00000100:00100000:0.0:1785220581.601581:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927424 00010000:00010000:0.0:1785220581.601593:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ddf6b984/0xf62499ab927866e2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7acf3 expref: 382 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601604:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927680 00000100:00100000:0.0:1785220581.601607:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927680 00010000:00010000:0.0:1785220581.601618:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009c104a5f/0xf62499ab9278638c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab48 expref: 383 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601629:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519927936 00000100:00100000:0.0:1785220581.601632:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519927936 00010000:00010000:0.0:1785220581.601643:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fb1fed34/0xf62499ab92789232 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2a9 expref: 340 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601654:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928064 00000100:00100000:0.0:1785220581.601656:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928064 00010000:00010000:0.0:1785220581.601667:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c0190141/0xf62499ab92789152 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c224 expref: 341 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.601679:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100153 00000100:00100000:0.0:1785220581.601686:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+341:2407:x1871938519926784:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.601696:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.601702:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000070fe3c19/0xf62499ab92788d4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c041 expref: 341 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601712:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000070fe3c19/0xf62499ab92788d4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c041 expref: 341 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601729:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070fe3c19/0xf62499ab92788d4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c041 expref: 341 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601743:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.601747:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.601752:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601772:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601794:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.601828:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+340:2407:x1871938519926784:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 141us (3142us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.601836:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100153 00000100:00100000:0.0:1785220581.601852:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100154 00000100:00100000:0.0:1785220581.601859:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2407:x1871938519926656:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.601867:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.601873:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002d650261/0xf62499ab927860a6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9c7 expref: 383 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601882:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002d650261/0xf62499ab927860a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9c7 expref: 383 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601896:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.601899:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.601903:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601911:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601923:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.601941:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+383:2407:x1871938519926656:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (3160us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.601948:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100154 00000100:00100000:0.0:1785220581.601962:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100155 00000100:00100000:0.0:1785220581.601968:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+339:2407:x1871938519926912:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.601975:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.601980:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000515aacde/0xf62499ab92788c51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bfca expref: 339 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.601989:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000515aacde/0xf62499ab92788c51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bfca expref: 339 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602001:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000515aacde/0xf62499ab92788c51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bfca expref: 339 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602011:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.602013:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.602017:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602024:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602042:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.602060:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+338:2407:x1871938519926912:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (3254us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.602066:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100155 00000100:00100000:0.0:1785220581.602121:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100156 00000100:00100000:0.0:1785220581.602129:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2407:x1871938519927040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.602136:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.602142:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005433698e/0xf62499ab92788f8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c14b expref: 337 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602151:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005433698e/0xf62499ab92788f8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c14b expref: 337 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602162:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005433698e/0xf62499ab92788f8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c14b expref: 337 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602171:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.602173:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.602177:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602185:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.602207:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.602342:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510639616:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.602356:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602377:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602385:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602394:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602402:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602418:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.602568:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1148:3652:x1871938510636288:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13742us (15443us total) trans 4295070519 rc 0/0 00000100:00100000:2.0:1785220581.602576:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284222 00010000:00010000:2.0:1785220581.602598:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.602603:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602612:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.602616:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602625:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602637:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602644:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4a98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.602661:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.602663:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.602664:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.602691:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ec700/2, rc 0, transno 4295070519 00000100:00100000:2.0:1785220581.602702:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510636288:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.602772:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.602785:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510640128:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.602818:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.603112:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+336:2407:x1871938519927040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 981us (4121us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.603119:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100156 00000100:00100000:0.0:1785220581.603138:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100157 00000100:00100000:0.0:1785220581.603147:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+335:2407:x1871938519927296:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.603156:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.603161:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ceb8d68a/0xf62499ab92788eb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0e9 expref: 335 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.603172:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ceb8d68a/0xf62499ab92788eb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0e9 expref: 335 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.603187:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8d68a/0xf62499ab92788eb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0e9 expref: 335 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.603198:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.603202:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.603206:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.603395:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100158 00000100:00100000:3.0:1785220581.603407:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+382:2407:x1871938519927552:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.603416:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.603422:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b1589c39/0xf62499ab92786592 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac4b expref: 382 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603432:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b1589c39/0xf62499ab92786592 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac4b expref: 382 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603446:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b1589c39/0xf62499ab92786592 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ac4b expref: 382 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603457:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603460:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603464:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603473:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603493:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.603512:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+381:2407:x1871938519927552:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (4485us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.603519:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100158 00000100:00100000:3.0:1785220581.603535:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100159 00000100:00100000:3.0:1785220581.603542:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+380:2407:x1871938519927808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.603549:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.603554:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000071202272/0xf62499ab92786481 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abc6 expref: 380 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603564:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000071202272/0xf62499ab92786481 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abc6 expref: 380 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603575:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000071202272/0xf62499ab92786481 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abc6 expref: 380 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603584:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603586:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603590:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603597:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603614:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.603632:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+379:2407:x1871938519927808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (4586us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.603639:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100159 00000100:00100000:3.0:1785220581.603651:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100160 00000100:00100000:3.0:1785220581.603658:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+378:2407:x1871938519927168:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.603666:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.603673:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ff8d1cdd/0xf62499ab9278624a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aaa7 expref: 378 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603689:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ff8d1cdd/0xf62499ab9278624a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aaa7 expref: 378 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603701:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ff8d1cdd/0xf62499ab9278624a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aaa7 expref: 378 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603710:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603713:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603716:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603723:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603738:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.603756:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+377:2407:x1871938519927168:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (4690us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.603775:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100160 00000100:00100000:3.0:1785220581.603789:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100161 00000100:00100000:3.0:1785220581.603797:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+376:2407:x1871938519927424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.603804:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.603809:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ddf6b984/0xf62499ab927866e2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7acf3 expref: 376 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603824:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ddf6b984/0xf62499ab927866e2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7acf3 expref: 376 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603836:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddf6b984/0xf62499ab927866e2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7acf3 expref: 376 pid: 14166 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603846:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603849:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603853:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603860:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603878:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.603895:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+375:2407:x1871938519927424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (4813us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.603901:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100161 00000100:00100000:3.0:1785220581.603914:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100162 00000100:00100000:3.0:1785220581.603921:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e12c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+374:2407:x1871938519927680:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.603928:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.603934:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009c104a5f/0xf62499ab9278638c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab48 expref: 374 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603948:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009c104a5f/0xf62499ab9278638c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab48 expref: 374 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603962:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c104a5f/0xf62499ab9278638c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ab48 expref: 374 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603971:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603974:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.603978:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.603985:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.604004:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.604022:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e12c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+373:2407:x1871938519927680:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (4893us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.604029:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100162 00000100:00100000:3.0:1785220581.604041:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100163 00000100:00100000:3.0:1785220581.604048:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+334:2404:x1871938519927936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.604071:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.604077:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fb1fed34/0xf62499ab92789232 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2a9 expref: 334 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.604113:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fb1fed34/0xf62499ab92789232 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2a9 expref: 334 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.604126:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb1fed34/0xf62499ab92789232 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2a9 expref: 334 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.604136:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7221:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.604140:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7221:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.604143:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.604151:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.604168:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.604187:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+333:2404:x1871938519927936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 139us (3219us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.604193:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100163 00000100:00100000:3.0:1785220581.604208:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100164 00010000:00010000:0.0:1785220581.604309:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.604332:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.604353:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+332:2407:x1871938519927296:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1208us (5344us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.604422:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604432:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604441:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604449:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604466:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.604621:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1080:3653:x1871938510638208:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14272us (14630us total) trans 4295069930 rc 0/0 00000100:00100000:2.0:1785220581.604629:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284229 00000100:00100000:2.0:1785220581.604646:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510640128 00000100:00100000:2.0:1785220581.604652:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510640128 00000100:00100000:2.0:1785220581.604683:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284236 00000100:00100000:2.0:1785220581.604689:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1148:3655:x1871938510640128:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.604712:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29219 count 1 00010000:00010000:2.0:1785220581.604732:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604746:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005d862ce1/0xf62499ab92789343 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c34a expref: 332 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604752:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7223:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.604755:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7223:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.604758:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604795:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005d862ce1/0xf62499ab92789343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c34a expref: 332 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604811:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005d862ce1/0xf62499ab92789343 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c34a expref: 332 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.604823:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510640256:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.604865:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896780 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.604884:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604893:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604901:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604907:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.604922:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.605019:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1080:3653:x1871938510637312:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15325us (15669us total) trans 4295069931 rc 0/0 00000100:00100000:2.0:1785220581.605026:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284225 00010000:00010000:2.0:1785220581.605046:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605060:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605068:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605076:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605110:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.605196:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1149:3655:x1871938510636544:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15138us (15284us total) trans 4295070520 rc 0/0 00000100:00100000:2.0:1785220581.605203:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284227 00000100:00100000:3.0:1785220581.605446:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+332:2404:x1871938519928064:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.605456:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.605463:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c0190141/0xf62499ab92789152 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c224 expref: 332 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.605478:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c0190141/0xf62499ab92789152 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c224 expref: 332 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.605494:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0190141/0xf62499ab92789152 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c224 expref: 332 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.605507:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7220:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.605510:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7220:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.605515:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.605523:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.605543:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.605559:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+331:2404:x1871938519928064:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 119us (4565us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.605653:0:14258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d650261/0xf62499ab927860a6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a9c7 expref: 372 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.605662:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510636928:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.605677:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.605683:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.605687:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.605692:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.605696:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.605709:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:1.0:1785220581.605794:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1080:3653:x1871938510635904:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17380us (18781us total) trans 4295069932 rc 0/0 00000100:00100000:1.0:1785220581.605801:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284220 00010000:00010000:2.0:1785220581.605829:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605901:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605909:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605916:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.605929:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1785220581.606013:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1149:3652:x1871938510639488:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14648us (14984us total) trans 4295070521 rc 0/0 00000100:00100000:2.0:1785220581.606019:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284232 00010000:00010000:2.0:1785220581.606034:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1785220581.606040:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606047:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.606051:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606060:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606072:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606080:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4ac9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606125:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220581.606137:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606149:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.606152:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.606157:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606160:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.606166:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606171:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.606175:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606178:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4aad lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.606183:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606190:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.606192:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220581.606193:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606199:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.606201:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606208:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606218:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 1/0,0 mode: --/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606225:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4abb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606237:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.606240:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606246:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.606249:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606255:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606265:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606273:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a4a8a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606286:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.606289:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606295:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.606298:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606306:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606315:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606322:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4ade lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606335:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220581.606335:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1149:3653:x1871938510636416:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17734us (19286us total) trans 4295070522 rc 0/0 00010000:00010000:2.0:1785220581.606339:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.606341:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284221 00010000:00010000:2.0:1785220581.606345:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.606349:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606356:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606365:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.606371:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606372:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a4a91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.606383:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002f3cbee5/0xf62499ab92789079 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c1d7 expref: 331 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.606384:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.606386:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.606388:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.606396:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x721f:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.606399:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.606402:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 4/0,1 mode: --/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.606415:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa3800/2, rc 0, transno 4295069930 00000100:00100000:2.0:1785220581.606425:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510638208:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.606426:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002f3cbee5/0xf62499ab92789079 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c1d7 expref: 331 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.606436:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e05180/2, rc 0, transno 4295069931 00010000:00010000:1.0:1785220581.606437:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002f3cbee5/0xf62499ab92789079 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c1d7 expref: 331 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.606440:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510637312:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.606447:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6eea00/2, rc 0, transno 4295069932 00000100:00100000:2.0:1785220581.606450:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510635904:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.606450:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510640384:192.168.201.51@tcp:104: 00000004:00080000:2.0:1785220581.606459:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ef800/2, rc 0, transno 4295070522 00000100:00100000:2.0:1785220581.606466:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510636416:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.606488:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99919180/2, rc 0, transno 4295070521 00000100:00100000:1.0:1785220581.606488:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.606492:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510639488:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.606690:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.606699:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfa5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510640640:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.606726:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.606786:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510640768:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.606805:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.606927:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100157 00000100:00100000:3.0:1785220581.606933:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100164 00010000:00010000:3.0:1785220581.606958:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.606967:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.606974:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.606975:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.606983:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.606985:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.606991:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.606997:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.606999:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.607005:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.607083:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e07100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1149:3655:x1871938510636672:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17515us (18047us total) trans 4295070523 rc 0/0 00000100:00100000:3.0:1785220581.607143:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1080:3654:x1871938510637952:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16516us (16563us total) trans 4295069933 rc 0/0 00000100:00100000:3.0:1785220581.607151:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284231 00000100:00100000:3.0:1785220581.607166:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510640640 00000100:00100000:3.0:1785220581.607171:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510640640 00000100:00100000:3.0:1785220581.607197:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510640768 00000100:00100000:3.0:1785220581.607200:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510640768 00000100:00100000:2.0:1785220581.607246:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510640512:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.607271:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.607348:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284224 00000100:00100000:3.0:1785220581.607376:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510640512 00000100:00100000:3.0:1785220581.607380:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510640512 00000100:00100000:2.0:1785220581.607388:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510641024:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.607404:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284237 00000100:00100000:2.0:1785220581.607408:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.607412:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1150:3653:x1871938510640640:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.607435:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29221 count 1 00000004:00080000:2.0:1785220581.607436:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ee680/2, rc 0, transno 4295070520 00000100:00100000:2.0:1785220581.607445:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636544:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.607453:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.607453:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ef100/2, rc 0, transno 4295070523 00000100:00100000:2.0:1785220581.607456:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636672:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.607465:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:3.0:1785220581.607466:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000037509ae3/0xf62499ab92789526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c446 expref: 329 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.607470:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510640896:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.607474:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7225:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.607477:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7225:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.607482:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607507:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000037509ae3/0xf62499ab92789526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c446 expref: 329 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.607507:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.607518:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000037509ae3/0xf62499ab92789526 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c446 expref: 329 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.607529:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510641152:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.607552:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.607558:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9780 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.607559:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510641280:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.607571:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.607579:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.607580:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607588:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607596:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.607600:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510641024 00000100:00100000:2.0:1785220581.607603:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510641024 00010000:00010000:3.0:1785220581.607605:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.607622:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510640896 00000100:00100000:2.0:1785220581.607625:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510640896 00000100:00100000:2.0:1785220581.607638:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284238 00000100:00100000:2.0:1785220581.607644:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1082:3653:x1871938510640768:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.607656:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29168 count 1 00010000:00010000:2.0:1785220581.607669:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.607681:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000193593fe/0xf62499ab927867e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad63 expref: 371 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.607684:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1083:3653:x1871938510638336:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17375us (17744us total) trans 4295069934 rc 0/0 00010000:00010000:2.0:1785220581.607688:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f0:0x0].0x0 since=29s v=(0 0) 00000100:00100000:3.0:1785220581.607690:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284228 00010000:00010000:2.0:1785220581.607692:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.607695:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.607702:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510641280 00000100:00100000:3.0:1785220581.607705:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510641280 00010000:00010000:2.0:1785220581.607718:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000193593fe/0xf62499ab927867e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad63 expref: 371 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.607725:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284239 00010000:00010000:2.0:1785220581.607728:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000193593fe/0xf62499ab927867e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad63 expref: 371 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.607730:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1151:3655:x1871938510640512:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.607739:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510641408:192.168.201.51@tcp:104: 00002000:00080000:3.0:1785220581.607740:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29220 count 1 00010000:00010000:3.0:1785220581.607751:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.607757:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.607795:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284240 00000100:00100000:1.0:1785220581.607800:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284241 00000100:00100000:2.0:1785220581.607802:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1082:3655:x1871938510641024:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.607807:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1082:3655:x1871938510640896:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.607813:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29170 count 1 00002000:00080000:1.0:1785220581.607816:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29169 count 1 00010000:00010000:2.0:1785220581.607826:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.607826:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.607837:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d752c7ae/0xf62499ab927868be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ada2 expref: 371 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.607844:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f1:0x0].0x0 since=29s v=(0 0) 00010000:00010000:1.0:1785220581.607847:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.607851:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.607871:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d752c7ae/0xf62499ab927868be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ada2 expref: 371 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.607880:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d752c7ae/0xf62499ab927868be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ada2 expref: 371 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.607890:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510641536:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.607910:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.607916:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284242 00000100:00100000:0.0:1785220581.607925:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1151:3653:x1871938510641280:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.607935:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29222 count 1 00010000:00010000:0.0:1785220581.607947:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607949:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e695bc00/0xf62499ab9278942a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3c8 expref: 329 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607958:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7224:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.607961:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7224:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.607961:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b0bba1ed/0xf62499ab9278960d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c493 expref: 329 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607966:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.607968:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7226:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.607972:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7226:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.607975:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607985:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e695bc00/0xf62499ab9278942a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3c8 expref: 329 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.607989:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b0bba1ed/0xf62499ab9278960d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c493 expref: 329 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.607995:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e695bc00/0xf62499ab9278942a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3c8 expref: 329 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.607998:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b0bba1ed/0xf62499ab9278960d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c493 expref: 329 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.608005:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510641664:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.608007:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510641792:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.608027:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26240 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.608031:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.608042:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.608046:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608048:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.608056:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608062:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.608064:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608066:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608075:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.608077:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608145:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608153:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4ad7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608172:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.608175:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608181:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.608184:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608190:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608198:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608205:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4aa6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.608291:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.608294:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.608298:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608304:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.608307:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608314:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608324:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608331:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4ac2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608344:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.608347:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608354:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.608357:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608364:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.608371:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1083:3653:x1871938510637824:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18325us (18481us total) trans 4295069935 rc 0/0 00010000:00010000:3.0:1785220581.608375:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.608377:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284226 00010000:00010000:3.0:1785220581.608381:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4ab4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608402:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.608404:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.608406:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.608412:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.608416:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.608425:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608434:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608443:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608449:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608479:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.608490:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fae6a886/0xf62499ab927869eb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae51 expref: 371 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.608497:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f2:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.608500:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.608504:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.608525:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fae6a886/0xf62499ab927869eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae51 expref: 371 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.608539:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fae6a886/0xf62499ab927869eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae51 expref: 371 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.608550:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510641920:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.608575:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.608606:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa2d80/2, rc 0, transno 4295069934 00000100:00100000:3.0:1785220581.608608:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1152:3655:x1871938510639360:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17112us (17325us total) trans 4295070524 rc 0/0 00000100:00100000:3.0:1785220581.608614:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284233 00000100:00100000:2.0:1785220581.608615:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510638336:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.608624:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa61500/2, rc 0, transno 4295069935 00000100:00100000:2.0:1785220581.608628:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa61500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510637824:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.608651:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.608656:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608664:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:2.0:1785220581.608667:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510642048:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.608668:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608675:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608685:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.608693:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a4ae5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.608701:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.608706:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.608707:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.608709:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.608728:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ae680/2, rc 0, transno 4295069933 00000100:00100000:2.0:1785220581.608728:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510642048 00000100:00100000:2.0:1785220581.608732:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510642048 00000100:00100000:3.0:1785220581.608733:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510637952:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.608771:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284243 00000100:00100000:2.0:1785220581.608778:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1152:3653:x1871938510642048:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.608793:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29223 count 1 00010000:00010000:2.0:1785220581.608811:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.608813:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.608824:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e3c196a4/0xf62499ab92789709 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c503 expref: 329 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.608844:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99918380/2, rc 0, transno 4295070524 00000100:00100000:1.0:1785220581.608850:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510639360:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.608901:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7227:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.608905:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7227:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.608908:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.608923:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e3c196a4/0xf62499ab92789709 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c503 expref: 329 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.608929:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e3c196a4/0xf62499ab92789709 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c503 expref: 329 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.608936:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510642176:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.608949:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.609000:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510642304:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.609021:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.609026:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.609028:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510642560:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.609041:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220581.609046:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510642432:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.609049:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.609068:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.609162:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510642304 00000100:00100000:0.0:1785220581.609167:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510642304 00000100:00100000:0.0:1785220581.609200:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510642560 00000100:00100000:0.0:1785220581.609203:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510642560 00000100:00100000:0.0:1785220581.609302:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510642432 00000100:00100000:0.0:1785220581.609306:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510642432 00000100:00100000:2.0:1785220581.609312:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510642688:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.609323:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284244 00000100:00100000:0.0:1785220581.609329:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1153:3654:x1871938510642304:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.609335:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:0.0:1785220581.609340:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29224 count 1 00010000:00010000:0.0:1785220581.609353:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609365:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a29a5004/0xf62499ab927897c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c57a expref: 329 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609372:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7228:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.609374:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7228:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.609379:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609396:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a29a5004/0xf62499ab927897c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c57a expref: 329 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609404:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a29a5004/0xf62499ab927897c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c57a expref: 329 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.609409:0:9945:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1053664321649ns, next at 1054664319638ns, rc = 0 00000100:00100000:0.0:1785220581.609414:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510643072:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.609438:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.609453:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510642688 00000100:00100000:0.0:1785220581.609456:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510642688 00000100:00100000:0.0:1785220581.609478:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284245 00000100:00100000:2.0:1785220581.609479:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510637440:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.609484:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1085:3653:x1871938510642560:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.609494:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29172 count 1 00010000:00010000:2.0:1785220581.609496:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609507:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609517:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006e689061/0xf62499ab92786b50 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af15 expref: 371 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609524:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f4:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.609526:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.609530:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609551:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006e689061/0xf62499ab92786b50 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af15 expref: 371 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609559:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006e689061/0xf62499ab92786b50 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af15 expref: 371 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.609569:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510643200:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.609586:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.609608:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284246 00000100:00100000:0.0:1785220581.609615:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b0e0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1085:3654:x1871938510642432:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.609624:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00002000:00080000:0.0:1785220581.609626:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29171 count 1 00010000:00010000:0.0:1785220581.609637:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609648:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000df8df365/0xf62499ab92786a8c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aeb3 expref: 371 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.609655:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.609656:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f3:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.609659:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.609663:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609681:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000df8df365/0xf62499ab92786a8c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aeb3 expref: 371 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609695:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000df8df365/0xf62499ab92786a8c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aeb3 expref: 371 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.609705:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b109c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510643328:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.609722:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.609737:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284247 00000100:00100000:0.0:1785220581.609743:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1085:3653:x1871938510642688:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.609754:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29173 count 1 00010000:00010000:0.0:1785220581.609779:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609790:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb6b63fb/0xf62499ab92786c7d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af93 expref: 371 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609797:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f5:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.609799:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.609803:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609819:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eb6b63fb/0xf62499ab92786c7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af93 expref: 371 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.609830:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eb6b63fb/0xf62499ab92786c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af93 expref: 371 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.609843:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b109880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510643456:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.609861:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.609887:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510638976:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.609900:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.610055:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928320 00000100:00100000:1.0:1785220581.610060:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928320 00010000:00010000:1.0:1785220581.610080:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004d8cd115/0xf62499ab927864b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abf0 expref: 372 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.610117:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928192 00000100:00100000:1.0:1785220581.610120:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928192 00010000:00010000:1.0:1785220581.610133:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001d5fa4c4/0xf62499ab92786178 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa30 expref: 373 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.610152:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100165 00000100:00100000:1.0:1785220581.610159:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+373:2407:x1871938519928320:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.610168:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.610173:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004d8cd115/0xf62499ab927864b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abf0 expref: 373 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610181:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004d8cd115/0xf62499ab927864b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abf0 expref: 373 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610198:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d8cd115/0xf62499ab927864b9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7abf0 expref: 373 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610375:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.610379:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.610385:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.610391:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.610394:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610413:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.610431:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+372:2407:x1871938519928320:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 273us (806us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.610439:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100165 00000100:00100000:1.0:1785220581.610452:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928448 00000100:00100000:1.0:1785220581.610455:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928448 00010000:00010000:1.0:1785220581.610467:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005bf7ab59/0xf62499ab927892cc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c30b expref: 330 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.610487:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100166 00000100:00100000:1.0:1785220581.610494:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+371:2407:x1871938519928192:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.610501:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.610506:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001d5fa4c4/0xf62499ab92786178 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa30 expref: 371 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610515:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001d5fa4c4/0xf62499ab92786178 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa30 expref: 371 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610529:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d5fa4c4/0xf62499ab92786178 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aa30 expref: 371 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610539:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.610542:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.610547:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610554:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610573:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.610591:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+370:2407:x1871938519928192:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (935us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.610597:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100166 00000100:00100000:1.0:1785220581.610610:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100167 00000100:00100000:1.0:1785220581.610617:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+330:2407:x1871938519928448:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.610627:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.610631:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005bf7ab59/0xf62499ab927892cc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c30b expref: 330 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610641:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005bf7ab59/0xf62499ab927892cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c30b expref: 330 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610655:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7222:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.610658:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7222:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.610661:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610669:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.610680:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.610699:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+330:2407:x1871938519928448:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.610704:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100167 00000100:00100000:1.0:1785220581.610748:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510642944:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1785220581.610812:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.610837:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510642944 00000100:00100000:1.0:1785220581.610841:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510642944 00000100:00100000:1.0:1785220581.610866:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:1.0:1785220581.610872:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+1086:3655:x1871938510642944:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1785220581.610885:0:8444:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 315392 granted 582591936 pending 0 free 3891458048 avail 3659935744 00000100:00100000:1.0:1785220581.610918:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+1086:3655:x1871938510642944:12345-0@lo:13:osp-pre-1-0.0 Request processed in 45us (105us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.610923:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00100000:1.0:1785220581.610959:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510642944:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1785220581.610977:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.610983:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1871938510642816:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1785220581.611009:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.611031:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510642816 00000100:00100000:1.0:1785220581.611034:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510642816 00000100:00100000:1.0:1785220581.611061:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12554 00000100:00100000:1.0:1785220581.611067:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1871938510642816:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1785220581.611112:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1439215616 avail 1292976128 00000100:00100000:1.0:1785220581.611143:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1871938510642816:12345-0@lo:41:osp-pre-0-1.0 Request processed in 76us (133us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.611149:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12554 00000100:00100000:1.0:1785220581.611181:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1871938510642816:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1785220581.613642:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.613662:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.613674:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.613684:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.613705:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.613915:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1086:3655:x1871938510638080:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23361us (23444us total) trans 4295069936 rc 0/0 00000100:00100000:0.0:1785220581.613926:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284230 00010000:00010000:0.0:1785220581.614107:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.614114:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.614124:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.614128:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.614137:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.614160:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.614168:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4ad0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.614187:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.614189:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.614191:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.614437:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa1c00/2, rc 0, transno 4295069936 00000100:00100000:1.0:1785220581.614453:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510638080:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.616291:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.616311:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.616323:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.616330:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.616366:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.616633:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.616641:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.616650:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.616654:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.616668:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.616689:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.616697:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4a9f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.616714:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.616716:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.616718:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.616769:0:14250:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bf7ab59/0xf62499ab927892cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c30b expref: 329 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.616783:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510640000:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.616808:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.616817:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.616824:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.616832:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.616839:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.616894:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.616995:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.617001:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.617009:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.617012:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.617021:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.617032:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.617039:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4aec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.617053:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.617055:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.617057:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.617539:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510643584:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.618129:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1154:3653:x1871938510639872:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17639us (18089us total) trans 4295070525 rc 0/0 00000100:00100000:2.0:1785220581.618129:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1086:3655:x1871938510636032:12345-0@lo:6:osp-syn-1-0.0 Request processed in 28799us (28851us total) trans 4295069937 rc 0/0 00000100:00100000:2.0:1785220581.618143:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284223 00000100:00100000:3.0:1785220581.618200:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284235 00000004:00080000:2.0:1785220581.618211:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725c00/2, rc 0, transno 4295070525 00000100:00100000:2.0:1785220581.618224:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510639872:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.618311:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510643712:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.618348:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.618493:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510640256:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.618513:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.618819:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510643072:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.618850:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.618901:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510643200:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.618914:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.618961:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b109c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510643328:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.618973:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619017:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b109880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510643456:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.619027:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.619279:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510641408:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.619292:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.619328:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510642176:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.619340:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619488:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510641792:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.619503:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619581:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.619604:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510641920:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.619605:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.619616:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619622:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.619640:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510641664:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.619640:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.619655:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619656:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.619672:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.619702:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.619711:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929728 00000100:00100000:2.0:1785220581.619717:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929728 00000100:00100000:0.0:1785220581.619719:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.619736:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.619746:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d752c7ae/0xf62499ab927868be lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ada2 expref: 370 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619750:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929984 00000100:00100000:0.0:1785220581.619753:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929984 00010000:00010000:0.0:1785220581.619780:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e695bc00/0xf62499ab9278942a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3c8 expref: 329 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.619781:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929600 00000100:00100000:2.0:1785220581.619784:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929600 00010000:00010000:2.0:1785220581.619796:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb6b63fb/0xf62499ab92786c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af93 expref: 371 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619799:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929856 00000100:00100000:0.0:1785220581.619801:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929856 00000100:00100000:2.0:1785220581.619812:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519930112 00010000:00010000:0.0:1785220581.619812:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b0bba1ed/0xf62499ab9278960d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c493 expref: 330 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.619814:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519930112 00010000:00010000:2.0:1785220581.619824:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fae6a886/0xf62499ab927869eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae51 expref: 372 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.619845:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928576 00000100:00100000:2.0:1785220581.619848:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928576 00010000:00010000:2.0:1785220581.619859:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005d862ce1/0xf62499ab92789343 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c34a expref: 331 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.619876:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928832 00000100:00100000:2.0:1785220581.619878:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928832 00000100:00100000:0.0:1785220581.619888:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929088 00010000:00010000:2.0:1785220581.619889:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000037509ae3/0xf62499ab92789526 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c446 expref: 332 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619890:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929088 00000100:00100000:2.0:1785220581.619901:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929344 00010000:00010000:0.0:1785220581.619902:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e3c196a4/0xf62499ab92789709 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c503 expref: 333 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.619903:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929344 00010000:00010000:2.0:1785220581.619913:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006e689061/0xf62499ab92786b50 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af15 expref: 373 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.619917:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100168 00000100:00100000:0.0:1785220581.619924:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+373:2406:x1871938519929728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.619925:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100169 00000100:00100000:2.0:1785220581.619929:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce7bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+333:2406:x1871938519929984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.619937:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.619937:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.619943:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e695bc00/0xf62499ab9278942a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3c8 expref: 333 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.619943:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d752c7ae/0xf62499ab927868be lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ada2 expref: 373 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.619955:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e695bc00/0xf62499ab9278942a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3c8 expref: 333 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.619955:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d752c7ae/0xf62499ab927868be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ada2 expref: 373 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.619977:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e695bc00/0xf62499ab9278942a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3c8 expref: 333 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.619979:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.619984:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.619989:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7224:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.619990:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.619992:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7224:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.619996:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.619999:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620003:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620016:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.620018:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.620055:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce7bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+332:2406:x1871938519929984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 125us (449us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.620062:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100169 00000100:00100000:2.0:1785220581.620079:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100170 00000100:00100000:3.0:1785220581.620184:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510641152:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.620199:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.620229:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+373:2406:x1871938519929600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.620239:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.620244:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eb6b63fb/0xf62499ab92786c7d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af93 expref: 373 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620254:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eb6b63fb/0xf62499ab92786c7d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af93 expref: 373 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620268:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb6b63fb/0xf62499ab92786c7d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af93 expref: 373 pid: 14257 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620283:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.620286:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.620290:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620297:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620315:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.620339:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100171 00000100:00100000:3.0:1785220581.620348:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+331:2406:x1871938519929856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.620359:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.620364:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b0bba1ed/0xf62499ab9278960d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c493 expref: 331 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.620376:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b0bba1ed/0xf62499ab9278960d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c493 expref: 331 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.620391:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0bba1ed/0xf62499ab9278960d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c493 expref: 331 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.620406:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7226:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.620409:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7226:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.620413:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.620420:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.620435:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.620500:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510643712 00000100:00100000:1.0:1785220581.620505:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510643712 00000100:00100000:1.0:1785220581.620533:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284248 00000100:00100000:1.0:1785220581.620538:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1154:3653:x1871938510643712:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.620561:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29225 count 1 00010000:00010000:1.0:1785220581.620580:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.620593:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fea0b1b6/0xf62499ab927898ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5c7 expref: 330 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.620599:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7229:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.620603:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7229:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.620606:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.620628:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fea0b1b6/0xf62499ab927898ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5c7 expref: 330 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.620639:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fea0b1b6/0xf62499ab927898ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5c7 expref: 330 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.620652:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99066300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510643840:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.620726:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+372:2406:x1871938519929600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 640us (1101us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.620726:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+372:2406:x1871938519929728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 800us (1144us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.620734:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100170 00000100:00100000:0.0:1785220581.620738:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100168 00000100:00100000:2.0:1785220581.620748:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100172 00000100:00100000:2.0:1785220581.620754:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+370:2406:x1871938519930112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.620772:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.620777:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fae6a886/0xf62499ab927869eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae51 expref: 370 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.620784:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100173 00010000:00010000:2.0:1785220581.620787:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fae6a886/0xf62499ab927869eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae51 expref: 370 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.620791:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+330:2404:x1871938519928576:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.620798:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.620800:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fae6a886/0xf62499ab927869eb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ae51 expref: 370 pid: 8441 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620803:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005d862ce1/0xf62499ab92789343 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c34a expref: 330 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620813:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005d862ce1/0xf62499ab92789343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c34a expref: 330 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620816:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.620820:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.620823:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620825:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d862ce1/0xf62499ab92789343 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c34a expref: 330 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620831:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620846:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7223:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.620848:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.620850:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7223:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.620854:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620861:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.620865:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+369:2406:x1871938519930112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (1208us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.620870:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100172 00010000:00010000:0.0:1785220581.620878:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.620884:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100174 00000100:00100000:2.0:1785220581.620890:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+329:2404:x1871938519928832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.620894:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+329:2404:x1871938519928576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (1222us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.620897:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220581.620900:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100173 00010000:00010000:2.0:1785220581.620902:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000037509ae3/0xf62499ab92789526 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c446 expref: 329 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620911:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000037509ae3/0xf62499ab92789526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c446 expref: 328 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.620911:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100175 00000100:00100000:0.0:1785220581.620917:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+328:2404:x1871938519929088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.620923:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.620924:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037509ae3/0xf62499ab92789526 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c446 expref: 328 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620928:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e3c196a4/0xf62499ab92789709 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c503 expref: 328 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620937:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e3c196a4/0xf62499ab92789709 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c503 expref: 328 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620939:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7225:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.620942:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7225:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.620946:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620948:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e3c196a4/0xf62499ab92789709 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c503 expref: 327 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620955:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.620958:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7227:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.620961:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7227:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.620965:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.620969:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.620972:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.620985:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+326:2404:x1871938519928832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1283us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.620986:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.620991:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100174 00000100:00100000:0.0:1785220581.620999:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+325:2404:x1871938519929088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (1279us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.621003:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100176 00000100:00100000:0.0:1785220581.621006:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100175 00000100:00100000:2.0:1785220581.621009:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+368:2404:x1871938519929344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.621016:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.621021:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006e689061/0xf62499ab92786b50 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af15 expref: 368 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621025:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621030:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006e689061/0xf62499ab92786b50 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af15 expref: 368 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621034:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621042:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006e689061/0xf62499ab92786b50 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7af15 expref: 368 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621049:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621057:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.621058:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621060:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.621064:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621070:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621073:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.621209:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.621229:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+367:2404:x1871938519929344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 220us (1491us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.621236:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100176 00010000:00010000:2.0:1785220581.621256:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621265:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621274:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621281:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621295:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.621391:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1087:3655:x1871938510641024:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13589us (13980us total) trans 4295069939 rc 0/0 00000100:00100000:2.0:1785220581.621399:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284240 00010000:00010000:2.0:1785220581.621419:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621428:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621435:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621442:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621476:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.621488:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1086:3653:x1871938510642688:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11744us (12151us total) trans 4295069938 rc 0/0 00000100:00100000:0.0:1785220581.621495:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284247 00010000:00010000:0.0:1785220581.621518:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.621524:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621532:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.621536:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621545:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621557:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.621559:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1155:3653:x1871938510642048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12781us (12857us total) trans 4295070526 rc 0/0 00000100:00100000:2.0:1785220581.621565:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284243 00010000:00010000:0.0:1785220581.621565:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4b47 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621582:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.621585:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621591:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4de00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.621594:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621600:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.621602:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb22300/2, rc 0, transno 4295069938 00010000:00010000:0.0:1785220581.621609:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.621614:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510642688:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.621617:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4b1d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.621625:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb20e00/2, rc 0, transno 4295070526 00010000:00010000:0.0:1785220581.621629:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1785220581.621633:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510642048:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.621633:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621639:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4da00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.621642:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621647:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.621654:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621659:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621663:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621671:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4b2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621672:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621680:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621683:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.621684:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.621686:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.621688:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621695:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621700:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.621703:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621710:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621717:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621728:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.621779:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1155:3655:x1871938510640512:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14048us (14507us total) trans 4295070527 rc 0/0 00000100:00100000:2.0:1785220581.621786:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284239 00000100:00100000:0.0:1785220581.621811:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1086:3653:x1871938510642560:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12326us (12760us total) trans 4295069940 rc 0/0 00000004:00080000:2.0:1785220581.621815:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb23800/2, rc 0, transno 4295069940 00000100:00100000:0.0:1785220581.621817:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284245 00000100:00100000:2.0:1785220581.621819:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510642560:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.621828:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.621833:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621840:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.621842:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.621844:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621846:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621852:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621853:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.621856:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621864:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621865:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621873:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a4b0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621874:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.621881:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4b39 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.621888:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.621890:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.621891:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.621894:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.621896:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.621897:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.621898:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.621928:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.621948:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928704 00000100:00100000:0.0:1785220581.621953:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928704 00010000:00010000:0.0:1785220581.621970:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002f3cbee5/0xf62499ab92789079 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c1d7 expref: 325 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.621978:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.621988:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510644224:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.621996:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100177 00000100:00100000:0.0:1785220581.622002:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+325:2404:x1871938519928704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.622010:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.622015:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.622016:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002f3cbee5/0xf62499ab92789079 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c1d7 expref: 325 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.622025:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002f3cbee5/0xf62499ab92789079 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c1d7 expref: 325 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.622039:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.622042:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x721f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.622047:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.622055:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.622067:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.622081:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+325:2404:x1871938519928704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (153us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.622104:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100177 00000100:00100000:2.0:1785220581.622235:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510644224 00000100:00100000:2.0:1785220581.622240:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510644224 00000100:00100000:2.0:1785220581.622264:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284249 00000100:00100000:2.0:1785220581.622271:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1086:3653:x1871938510644224:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.622287:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29176 count 1 00010000:00010000:2.0:1785220581.622303:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622315:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000044f32dd/0xf62499ab92786f9b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b10d expref: 366 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622322:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f8:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.622325:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.622329:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622348:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000044f32dd/0xf62499ab92786f9b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b10d expref: 366 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.622348:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.622360:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000044f32dd/0xf62499ab92786f9b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b10d expref: 366 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.622371:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510644352:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.622377:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.622405:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.622422:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622430:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.622431:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b109c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+324:2406:x1871938519929856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2083us (2788us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.622438:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100171 00010000:00010000:2.0:1785220581.622439:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622446:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622464:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.622465:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519928960 00000100:00100000:3.0:1785220581.622468:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519928960 00010000:00010000:3.0:1785220581.622484:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000193593fe/0xf62499ab927867e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad63 expref: 367 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.622502:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929216 00000100:00100000:3.0:1785220581.622505:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929216 00010000:00010000:3.0:1785220581.622517:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a29a5004/0xf62499ab927897c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c57a expref: 324 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.622536:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100178 00000100:00100000:3.0:1785220581.622542:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+367:2404:x1871938519928960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.622543:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb23480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1155:3653:x1871938510641280:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14620us (14963us total) trans 4295070528 rc 0/0 00010000:00010000:3.0:1785220581.622549:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.622552:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284242 00010000:00010000:3.0:1785220581.622554:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000193593fe/0xf62499ab927867e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad63 expref: 367 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622563:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000193593fe/0xf62499ab927867e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad63 expref: 367 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622577:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000193593fe/0xf62499ab927867e5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ad63 expref: 367 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.622579:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb22680/2, rc 0, transno 4295070528 00000100:00100000:2.0:1785220581.622586:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510641280:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.622589:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.622592:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.622596:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.622601:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100179 00010000:00010000:3.0:1785220581.622604:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.622608:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+324:2404:x1871938519929216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.622621:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.622635:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+366:2404:x1871938519928960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (288us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.622638:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220581.622642:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100178 00010000:00010000:2.0:1785220581.622643:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a29a5004/0xf62499ab927897c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c57a expref: 324 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622652:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a29a5004/0xf62499ab927897c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c57a expref: 324 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622661:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622665:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a29a5004/0xf62499ab927897c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c57a expref: 324 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622670:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622674:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7228:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.622677:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7228:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.622682:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622683:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622690:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622691:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622703:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.622705:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.622719:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+323:2404:x1871938519929216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (341us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.622725:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100179 00010000:00010000:2.0:1785220581.622746:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622754:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622779:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622786:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.622794:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1087:3653:x1871938510640768:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15150us (15988us total) trans 4295069941 rc 0/0 00010000:00010000:2.0:1785220581.622797:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.622800:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284238 00010000:00010000:3.0:1785220581.622816:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220581.622820:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622827:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.622843:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622851:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622862:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.622863:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1155:3655:x1871938510640128:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18174us (20045us total) trans 4295070529 rc 0/0 00010000:00010000:3.0:1785220581.622870:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4b24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.622870:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284236 00010000:00010000:3.0:1785220581.622884:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.622887:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622893:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.622894:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba7800/2, rc 0, transno 4295069941 00010000:00010000:3.0:1785220581.622897:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.622902:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510640768:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.622904:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622912:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.622914:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622916:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622922:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4b08 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622923:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.622927:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622934:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622936:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.622938:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.622939:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.622945:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622948:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622952:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4af3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622956:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622966:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.622967:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.622969:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.622969:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.622975:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.622987:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.623004:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510644480:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.623027:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.623053:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfa5aa00/2, rc 0, transno 4295070530 00000100:00100000:3.0:1785220581.623054:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1155:3653:x1871938510640640:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15642us (16326us total) trans 4295070530 rc 0/0 00000100:00100000:3.0:1785220581.623060:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284237 00000100:00100000:2.0:1785220581.623060:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfa5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510640640:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.623071:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510644480 00000100:00100000:3.0:1785220581.623075:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510644480 00000100:00100000:3.0:1785220581.623113:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284250 00000100:00100000:3.0:1785220581.623120:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1155:3653:x1871938510644480:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.623207:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29227 count 1 00010000:00010000:3.0:1785220581.623287:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623300:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005e3da232/0xf62499ab92789a58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6ae expref: 322 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623306:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x722b:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.623309:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.623312:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623336:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005e3da232/0xf62499ab92789a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6ae expref: 322 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623347:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005e3da232/0xf62499ab92789a58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6ae expref: 322 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.623357:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510644736:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.623383:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.623399:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.623404:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623411:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.623414:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623421:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623428:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623433:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4b01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623448:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.623449:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.623450:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.623463:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510643968:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.623482:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.623495:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510644608:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.623506:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.623517:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1785220581.623521:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510644096:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.623532:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.623552:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510644864:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.623571:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.623572:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510643968 00000100:00100000:2.0:1785220581.623575:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510643968 00000100:00100000:2.0:1785220581.623597:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510644608 00000100:00100000:2.0:1785220581.623600:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510644608 00000100:00100000:2.0:1785220581.623626:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510644096 00000100:00100000:2.0:1785220581.623629:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510644096 00000100:00100000:2.0:1785220581.623644:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510644864 00000100:00100000:2.0:1785220581.623647:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510644864 00000100:00100000:2.0:1785220581.623665:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284251 00000100:00100000:2.0:1785220581.623672:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1157:3654:x1871938510643968:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.623687:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29226 count 1 00010000:00010000:2.0:1785220581.623698:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623710:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3cc10d4/0xf62499ab927899cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c653 expref: 322 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.623712:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.623716:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x722a:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.623719:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.623723:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623739:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d3cc10d4/0xf62499ab927899cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c653 expref: 322 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623748:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d3cc10d4/0xf62499ab927899cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c653 expref: 322 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.623757:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510644992:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.623758:0:14164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d752c7ae/0xf62499ab927868be lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ada2 expref: 365 pid: 14174 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.623781:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510641536:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.623788:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.623798:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.623802:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284252 00000100:00100000:2.0:1785220581.623808:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1157:3654:x1871938510644608:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.623808:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.623815:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.623819:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29228 count 1 00010000:00010000:2.0:1785220581.623829:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623845:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000090144a01/0xf62499ab92789b3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c709 expref: 322 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623852:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x722c:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.623855:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.623859:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623878:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000090144a01/0xf62499ab92789b3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c709 expref: 322 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623887:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000090144a01/0xf62499ab92789b3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c709 expref: 322 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.623898:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510645120:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.623904:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284253 00000100:00100000:3.0:1785220581.623911:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1088:3654:x1871938510644096:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.623916:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00002000:00080000:3.0:1785220581.623924:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29175 count 1 00000100:00100000:2.0:1785220581.623932:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284254 00010000:00010000:3.0:1785220581.623936:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.623938:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1088:3654:x1871938510644864:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.623948:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f62b03b5/0xf62499ab92786e52 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b07a expref: 364 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.623948:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29177 count 1 00010000:00010000:3.0:1785220581.623955:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f7:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.623957:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.623957:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623961:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623967:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002705f675/0xf62499ab927870eb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b145 expref: 364 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623979:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f62b03b5/0xf62499ab92786e52 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b07a expref: 364 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.623979:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f9:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.623982:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.623986:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.623989:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f62b03b5/0xf62499ab92786e52 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b07a expref: 364 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.623998:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510645248:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.624001:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002705f675/0xf62499ab927870eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b145 expref: 364 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624010:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002705f675/0xf62499ab927870eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b145 expref: 364 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.624015:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.624020:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510645376:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.624035:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.624047:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.624057:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.624076:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.624314:0:14163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f3cbee5/0xf62499ab92789079 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c1d7 expref: 322 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.624325:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510640384:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.624337:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624344:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.624345:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624349:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624357:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.624358:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624360:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624366:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624368:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624373:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624381:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624386:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.624388:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a4b16 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624393:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.624396:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624403:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.624406:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624408:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.624409:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.624411:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.624414:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624424:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 1/0,0 mode: --/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624432:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a4afa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624444:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.624446:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.624447:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.624511:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1089:3655:x1871938510640896:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16703us (17002us total) trans 4295069942 rc 0/0 00000100:00100000:2.0:1785220581.624513:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1158:3655:x1871938510639232:12345-0@lo:6:osp-syn-0-0.0 Request processed in 26466us (32963us total) trans 4295070531 rc 0/0 00000100:00100000:1.0:1785220581.624518:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284241 00000100:00100000:2.0:1785220581.624527:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284234 00000004:00080000:2.0:1785220581.624552:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba7480/2, rc 0, transno 4295069942 00000100:00100000:1.0:1785220581.624559:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.624571:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510640896:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.624583:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e05500/2, rc 0, transno 4295070527 00000100:00100000:2.0:1785220581.624592:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510640512:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.624639:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510645632:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.624662:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510645504:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.624663:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.624684:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.624695:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba6680/2, rc 0, transno 4295069939 00000100:00100000:2.0:1785220581.624701:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510641024:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.624710:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f726a00/2, rc 0, transno 4295070529 00000100:00100000:2.0:1785220581.624717:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510640128:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.624748:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510643584 00000004:00080000:2.0:1785220581.624751:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99918a80/2, rc 0, transno 4295070531 00000100:00100000:3.0:1785220581.624753:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510643584 00000100:00100000:2.0:1785220581.624757:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510639232:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.624796:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510645632 00000004:00080000:2.0:1785220581.624798:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ece00/2, rc 0, transno 4295069937 00000100:00100000:3.0:1785220581.624800:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510645632 00000100:00100000:2.0:1785220581.624801:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510636032:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.624811:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510645888:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.624821:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510645504 00000100:00100000:3.0:1785220581.624824:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510645504 00000100:00100000:2.0:1785220581.624848:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.624869:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.624874:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510645760:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.624894:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.624911:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510645888 00000100:00100000:2.0:1785220581.624913:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510645760 00000100:00100000:3.0:1785220581.624915:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510645888 00000100:00100000:2.0:1785220581.624916:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510645760 00000100:00100000:3.0:1785220581.624934:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284255 00000100:00100000:2.0:1785220581.624938:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284256 00000100:00100000:3.0:1785220581.624940:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1090:3652:x1871938510643584:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.624943:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1090:3653:x1871938510645632:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.624953:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29174 count 1 00002000:00080000:2.0:1785220581.624953:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29178 count 1 00010000:00010000:3.0:1785220581.624967:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624967:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624980:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000992a37c4/0xf62499ab92786d41 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b00a expref: 364 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624980:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000039a43e86/0xf62499ab92787185 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a7 expref: 364 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.624988:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71f6:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.624988:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71fa:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.624992:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.624992:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.624997:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.624997:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.625017:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000992a37c4/0xf62499ab92786d41 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b00a expref: 364 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625017:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000039a43e86/0xf62499ab92787185 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a7 expref: 364 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625028:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000039a43e86/0xf62499ab92787185 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a7 expref: 364 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.625032:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000992a37c4/0xf62499ab92786d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b00a expref: 364 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.625038:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510646016:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.625041:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510646144:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.625064:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.625079:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284257 00000100:00100000:2.0:1785220581.625101:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1160:3652:x1871938510645504:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.625113:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29229 count 1 00010000:00010000:2.0:1785220581.625125:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625136:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d903f05b/0xf62499ab92789c34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c795 expref: 321 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625143:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x722d:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.625146:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.625149:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625170:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d903f05b/0xf62499ab92789c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c795 expref: 321 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625179:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d903f05b/0xf62499ab92789c34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c795 expref: 321 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.625191:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510646272:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.625236:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284258 00000100:00100000:1.0:1785220581.625297:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1160:3655:x1871938510645888:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.625309:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29231 count 1 00010000:00010000:1.0:1785220581.625320:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625330:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003f5182a9/0xf62499ab92789ddf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c860 expref: 321 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625337:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x722f:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.625339:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.625344:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625358:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003f5182a9/0xf62499ab92789ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c860 expref: 321 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625366:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003f5182a9/0xf62499ab92789ddf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c860 expref: 321 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.625375:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510646400:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.625447:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.625447:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.625463:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284259 00000100:00100000:3.0:1785220581.625470:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1160:3655:x1871938510645760:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.625481:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29230 count 1 00010000:00010000:3.0:1785220581.625492:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.625503:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003c899da1/0xf62499ab92789cff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7fe expref: 321 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.625510:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x722e:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.625513:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.625517:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.625531:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003c899da1/0xf62499ab92789cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7fe expref: 321 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.625538:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003c899da1/0xf62499ab92789cff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7fe expref: 321 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.625548:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510646528:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.625561:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.625602:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.625610:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510646656:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.625645:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.625663:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510646656 00000100:00100000:2.0:1785220581.625667:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510646656 00000100:00100000:3.0:1785220581.625690:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510646784:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.625691:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284260 00000100:00100000:2.0:1785220581.625697:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1091:3653:x1871938510646656:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.625711:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:2.0:1785220581.625721:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29179 count 1 00010000:00010000:2.0:1785220581.625733:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625745:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000045a297e1/0xf62499ab92787242 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b217 expref: 364 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.625751:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71fb:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.625754:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.625758:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.625780:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.625789:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000045a297e1/0xf62499ab92787242 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b217 expref: 364 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.625792:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510646784 00000100:00100000:1.0:1785220581.625795:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510646784 00010000:00010000:2.0:1785220581.625798:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000045a297e1/0xf62499ab92787242 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b217 expref: 364 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.625807:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510646912:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.625826:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.625827:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284261 00000100:00100000:1.0:1785220581.625899:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1092:3653:x1871938510646784:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.625911:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29180 count 1 00010000:00010000:1.0:1785220581.625922:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625933:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b7cee91/0xf62499ab927872ce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b26b expref: 364 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625938:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71fc:0x0].0x0 since=29s v=(0 0) 00010000:00010000:1.0:1785220581.625940:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.625942:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625958:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b7cee91/0xf62499ab927872ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b26b expref: 364 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.625964:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b7cee91/0xf62499ab927872ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b26b expref: 364 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.625970:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510647040:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.625983:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.626047:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.626057:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.626064:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.626070:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.626074:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.626321:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.626351:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519929472 00000100:00100000:2.0:1785220581.626355:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519929472 00010000:00010000:2.0:1785220581.626374:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000df8df365/0xf62499ab92786a8c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aeb3 expref: 365 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.626397:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100180 00000100:00100000:2.0:1785220581.626403:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+365:2404:x1871938519929472:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.626412:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.626417:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000df8df365/0xf62499ab92786a8c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aeb3 expref: 365 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.626428:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000df8df365/0xf62499ab92786a8c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aeb3 expref: 365 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.626447:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df8df365/0xf62499ab92786a8c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7aeb3 expref: 365 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.626459:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.626462:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.626468:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.626477:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.626496:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.626514:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2404:x1871938519929472:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 110us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.626521:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100180 00010000:00010000:0.0:1785220581.626881:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.627037:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1161:3654:x1871938510642304:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17707us (18009us total) trans 4295070532 rc 0/0 00000100:00100000:0.0:1785220581.627044:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284244 00010000:00010000:0.0:1785220581.627077:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.627946:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.627957:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.627964:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.627995:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.628151:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b0e0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1093:3654:x1871938510642432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18536us (19081us total) trans 4295069943 rc 0/0 00000100:00100000:0.0:1785220581.628159:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284246 00000004:00080000:2.0:1785220581.628367:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb23100/2, rc 0, transno 4295069943 00000100:00100000:2.0:1785220581.628389:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510642432:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.628403:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb21880/2, rc 0, transno 4295070532 00000100:00100000:2.0:1785220581.628413:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510642304:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.628467:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.628480:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510647168:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.628509:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:2.0:1785220581.628511:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.628517:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.628526:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.628530:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.628538:0:14159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510647168 00010000:00010000:1.0:1785220581.628539:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.628543:0:14159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510647168 00010000:00010000:1.0:1785220581.628555:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.628563:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4b32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.628572:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284262 00000100:00100000:2.0:1785220581.628578:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1161:3655:x1871938510647168:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.628582:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.628585:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.628590:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.628592:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.628599:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.628600:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29232 count 1 00010000:00010000:1.0:1785220581.628609:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.628616:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.628616:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a4b40 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.628632:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee45469d/0xf62499ab92789eaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8c2 expref: 321 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.628637:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.628639:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.628640:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7230:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.628641:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.628644:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7230:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.628649:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.628673:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ee45469d/0xf62499ab92789eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8c2 expref: 321 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.628686:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ee45469d/0xf62499ab92789eaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8c2 expref: 321 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.628705:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510647296:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.628776:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.628878:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220581.628886:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b109880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510647424:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.628912:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.629311:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510647424 00000100:00100000:0.0:1785220581.629316:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510647424 00000100:00100000:0.0:1785220581.629351:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284263 00000100:00100000:0.0:1785220581.629359:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1093:3654:x1871938510647424:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.629373:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29181 count 1 00010000:00010000:0.0:1785220581.629392:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.629408:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000635ebaed/0xf62499ab927873d8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2db expref: 363 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.629415:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71fd:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.629418:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.629421:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.629441:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000635ebaed/0xf62499ab927873d8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2db expref: 363 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.629458:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000635ebaed/0xf62499ab927873d8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2db expref: 363 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.629469:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510647552:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.629493:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26cc0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.631012:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.631913:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.632351:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519930240 00000100:00100000:3.0:1785220581.632360:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519930240 00010000:00010000:3.0:1785220581.632396:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000044f32dd/0xf62499ab92786f9b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b10d expref: 364 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.632444:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100181 00000100:00100000:3.0:1785220581.632453:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb22a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2406:x1871938519930240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.632466:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.632473:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000044f32dd/0xf62499ab92786f9b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b10d expref: 364 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.632486:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000044f32dd/0xf62499ab92786f9b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b10d expref: 364 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.632512:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.632518:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.632524:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.632534:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.632556:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.632600:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb22a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2406:x1871938519930240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 147us (691us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.632608:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100181 00010000:00010000:2.0:1785220581.632909:0:14173:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000044f32dd/0xf62499ab92786f9b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b10d expref: 363 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.632924:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510644352:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.632945:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.632956:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.632964:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.632975:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.632982:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.632998:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.633144:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510644736:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.633160:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.633207:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1094:3653:x1871938510644224:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10935us (11189us total) trans 4295069944 rc 0/0 00000100:00100000:2.0:1785220581.633217:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284249 00000100:00100000:2.0:1785220581.633251:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510644992:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.633264:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.633280:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.633285:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.633292:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.633297:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.633306:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.633319:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.633326:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4b55 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.633342:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.633343:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.633345:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.633426:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99066300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510643840:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.633441:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.633507:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64a80/2, rc 0, transno 4295069944 00000100:00100000:3.0:1785220581.633527:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510644224:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.633984:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.633999:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510647680:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.634039:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.634586:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510645120:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.634604:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.634713:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510647680 00000100:00100000:2.0:1785220581.634718:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510647680 00000100:00100000:2.0:1785220581.634745:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284264 00000100:00100000:2.0:1785220581.634751:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1094:3653:x1871938510647680:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.634783:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29182 count 1 00010000:00010000:2.0:1785220581.634809:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.634824:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005c791710/0xf62499ab927874bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b34b expref: 362 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.634831:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71fe:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.634836:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.634840:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.634866:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005c791710/0xf62499ab927874bf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b34b expref: 362 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.634878:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005c791710/0xf62499ab927874bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b34b expref: 362 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.634889:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510647808:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.635457:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.638440:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.638473:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510645376:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.638491:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510646016:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.638499:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.638505:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.638548:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.638565:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.638573:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931008 00000100:00100000:2.0:1785220581.638578:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.638580:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931008 00000100:00100000:2.0:1785220581.638601:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.638614:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002705f675/0xf62499ab927870eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b145 expref: 363 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.638617:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519930368 00000100:00100000:2.0:1785220581.638620:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519930368 00010000:00010000:2.0:1785220581.638636:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005e3da232/0xf62499ab92789a58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6ae expref: 322 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.638643:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519930624 00000100:00100000:0.0:1785220581.638645:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519930624 00000100:00100000:2.0:1785220581.638653:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519930880 00000100:00100000:2.0:1785220581.638655:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519930880 00010000:00010000:0.0:1785220581.638656:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d3cc10d4/0xf62499ab927899cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c653 expref: 323 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.638665:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f62b03b5/0xf62499ab92786e52 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b07a expref: 364 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.638671:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931264 00000100:00100000:0.0:1785220581.638673:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931264 00000100:00100000:2.0:1785220581.638683:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100182 00010000:00010000:0.0:1785220581.638683:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000992a37c4/0xf62499ab92786d41 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b00a expref: 365 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.638690:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+365:2405:x1871938519931008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.638700:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100183 00010000:00010000:2.0:1785220581.638703:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220581.638705:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c307450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+323:2407:x1871938519930368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.638708:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002705f675/0xf62499ab927870eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b145 expref: 365 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.638712:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.638716:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005e3da232/0xf62499ab92789a58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6ae expref: 323 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.638719:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002705f675/0xf62499ab927870eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b145 expref: 365 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.638725:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005e3da232/0xf62499ab92789a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6ae expref: 323 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.638742:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002705f675/0xf62499ab927870eb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b145 expref: 365 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.638742:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e3da232/0xf62499ab92789a58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6ae expref: 323 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.638771:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.638771:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.638777:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.638777:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.638783:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.638783:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.638795:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.638795:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.638815:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.638816:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.638866:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100184 00000100:00100000:1.0:1785220581.638874:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+322:2407:x1871938519930624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.638883:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.638887:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d3cc10d4/0xf62499ab927899cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c653 expref: 322 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.638897:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d3cc10d4/0xf62499ab927899cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c653 expref: 322 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.638909:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3cc10d4/0xf62499ab927899cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c653 expref: 322 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.638918:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.638922:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722a:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.638923:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510645248:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.638926:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.638934:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.638939:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.638951:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.638997:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510646144:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.638997:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c307450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+321:2407:x1871938519930368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 290us (448us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.639005:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100183 00010000:00010000:3.0:1785220581.639011:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.639022:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100185 00000100:00100000:0.0:1785220581.639028:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2407:x1871938519930880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.639034:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100186 00010000:00010000:0.0:1785220581.639035:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220581.639040:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+364:2407:x1871938519931264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.639040:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f62b03b5/0xf62499ab92786e52 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b07a expref: 364 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.639046:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.639050:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f62b03b5/0xf62499ab92786e52 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b07a expref: 364 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.639051:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000992a37c4/0xf62499ab92786d41 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b00a expref: 364 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.639061:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000992a37c4/0xf62499ab92786d41 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b00a expref: 364 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.639064:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f62b03b5/0xf62499ab92786e52 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b07a expref: 364 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.639073:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000992a37c4/0xf62499ab92786d41 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b00a expref: 363 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.639073:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.639076:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.639081:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.639082:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.639118:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.639136:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.639282:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.639288:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.639297:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.639313:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.639378:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+362:2405:x1871938519931008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 687us (947us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.639385:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100182 00000100:00100000:1.0:1785220581.639395:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+320:2407:x1871938519930624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 523us (828us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.639401:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100184 00010000:00010000:1.0:1785220581.639425:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639434:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639443:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.639445:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.639451:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639466:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.639496:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.639511:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519930496 00000100:00100000:2.0:1785220581.639515:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519930496 00010000:00010000:2.0:1785220581.639530:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fea0b1b6/0xf62499ab927898ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5c7 expref: 320 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.639550:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519930752 00000100:00100000:2.0:1785220581.639552:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519930752 00010000:00010000:2.0:1785220581.639563:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000090144a01/0xf62499ab92789b3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c709 expref: 321 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.639581:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100187 00000100:00100000:2.0:1785220581.639587:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+321:2407:x1871938519930496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.639594:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.639598:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fea0b1b6/0xf62499ab927898ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5c7 expref: 321 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.639607:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fea0b1b6/0xf62499ab927898ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5c7 expref: 321 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.639621:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fea0b1b6/0xf62499ab927898ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5c7 expref: 321 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.639621:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1162:3654:x1871938510643968:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15949us (16136us total) trans 4295070533 rc 0/0 00000100:00100000:1.0:1785220581.639630:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284251 00010000:00010000:2.0:1785220581.639632:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7229:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.639634:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7229:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.639638:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.639647:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.639651:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100188 00000100:00100000:1.0:1785220581.639657:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+320:2407:x1871938519930752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.639662:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.639665:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.639669:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000090144a01/0xf62499ab92789b3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c709 expref: 320 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.639679:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+320:2407:x1871938519930496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (234us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.639679:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000090144a01/0xf62499ab92789b3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c709 expref: 320 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.639685:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100187 00010000:00010000:1.0:1785220581.639698:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090144a01/0xf62499ab92789b3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c709 expref: 319 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639709:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.639711:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.639715:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639723:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.639727:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.639740:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.639756:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+318:2407:x1871938519930752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.639775:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100188 00000100:00100000:1.0:1785220581.639787:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931136 00000100:00100000:1.0:1785220581.639789:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931136 00000100:00100000:2.0:1785220581.639792:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510646272:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.639802:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000039a43e86/0xf62499ab92787185 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a7 expref: 362 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.639805:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.639823:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100189 00000100:00100000:1.0:1785220581.639828:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+362:2407:x1871938519931136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.639836:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.639840:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000039a43e86/0xf62499ab92787185 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a7 expref: 362 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639850:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000039a43e86/0xf62499ab92787185 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a7 expref: 362 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639862:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039a43e86/0xf62499ab92787185 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a7 expref: 362 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639871:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.639874:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.639878:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639885:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639905:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.639912:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.639920:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.639926:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+361:2407:x1871938519931136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (199us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.639929:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.639932:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100189 00010000:00010000:2.0:1785220581.639936:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.639947:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220581.639951:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639958:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639965:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639972:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.639984:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220581.640045:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1096:3654:x1871938510644864:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16106us (16471us total) trans 4295069946 rc 0/0 00000100:00100000:2.0:1785220581.640046:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1096:3653:x1871938510645632:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15103us (15382us total) trans 4295069945 rc 0/0 00000100:00100000:1.0:1785220581.640052:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284254 00010000:00010000:1.0:1785220581.640069:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.640075:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640082:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d400 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:0.0:1785220581.640227:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+360:2407:x1871938519930880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1197us (1645us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.640235:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100185 00010000:00010000:0.0:1785220581.640256:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.640265:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.640274:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.640281:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.640295:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.640370:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1096:3654:x1871938510644096:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16459us (16834us total) trans 4295069947 rc 0/0 00000100:00100000:3.0:1785220581.640473:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+359:2407:x1871938519931264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1431us (1868us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.640479:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100186 00000100:00100000:3.0:1785220581.640518:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510646528:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.640532:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.640557:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.640565:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.640574:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.640581:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640585:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.640589:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:1.0:1785220581.640595:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640606:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640614:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ef55399/0xf62499ab927a4b63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640632:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.640636:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640643:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.640646:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640654:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.640664:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1097:3652:x1871938510643584:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15724us (16104us total) trans 4295069948 rc 0/0 00010000:00010000:1.0:1785220581.640665:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640672:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a4b7f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640684:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.640688:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640694:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.640697:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640704:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.640717:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284256 00010000:00010000:1.0:1785220581.640719:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640725:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000072eeceb1/0xf62499ab927a4b78 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.640736:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.640740:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.640747:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.640747:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.640750:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640751:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.640769:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640769:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.640772:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640780:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.640783:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640789:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.640790:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a4b71 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.640797:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4b86 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.640806:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.640807:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.640808:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.640809:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.640811:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.640813:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.640841:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa0380/2, rc 0, transno 4295069945 00000100:00100000:1.0:1785220581.640855:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510645632:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.640916:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510646400:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.640930:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.641368:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284253 00010000:00010000:0.0:1785220581.641395:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641402:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641410:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641418:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641441:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.641481:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.641486:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.641492:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.641496:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.641503:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.641515:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.641522:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4b4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.641536:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1162:3653:x1871938510643712:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20998us (23188us total) trans 4295070534 rc 0/0 00010000:00010000:2.0:1785220581.641541:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.641542:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.641544:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.641544:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284248 00010000:00010000:0.0:1785220581.641564:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641573:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641581:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641588:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641611:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.641685:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284255 00000100:00100000:0.0:1785220581.641687:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1162:3654:x1871938510644608:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17879us (18179us total) trans 4295070535 rc 0/0 00000100:00100000:0.0:1785220581.641693:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284252 00010000:00010000:3.0:1785220581.641708:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.641708:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.641713:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641719:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.641723:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641730:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.641732:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10aa00/2, rc 0, transno 4295069948 00000100:00100000:3.0:1785220581.641737:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510643584:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.641741:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641748:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a4b6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641774:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.641776:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.641778:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.641781:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.641788:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641796:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.641804:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.642422:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.642456:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.642473:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.642478:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.642485:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.642489:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.642497:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.642509:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.642515:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4b5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.642532:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.642533:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.642535:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.642551:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1162:3653:x1871938510644480:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19431us (19522us total) trans 4295070536 rc 0/0 00000100:00100000:0.0:1785220581.642559:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284250 00000100:00100000:3.0:1785220581.642652:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220581.642662:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510647936:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.642662:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510648064:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.642696:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.642738:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510647936 00000100:00100000:0.0:1785220581.642745:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510647936 00000100:00100000:0.0:1785220581.642790:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284265 00000100:00100000:0.0:1785220581.642799:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1095:3652:x1871938510647936:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.642818:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29183 count 1 00010000:00010000:0.0:1785220581.642835:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.642851:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005dbbc968/0xf62499ab9278753d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b398 expref: 358 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.642858:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x71ff:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.642862:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.642867:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.642893:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005dbbc968/0xf62499ab9278753d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b398 expref: 358 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.642902:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005dbbc968/0xf62499ab9278753d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b398 expref: 358 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.642912:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510648192:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.642961:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.642978:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.643017:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64700/2, rc 0, transno 4295069947 00000100:00100000:0.0:1785220581.643026:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510644096:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.643038:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb65c00/2, rc 0, transno 4295069946 00000100:00100000:0.0:1785220581.643042:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510644864:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.643054:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb67b80/2, rc 0, transno 4295070533 00000100:00100000:0.0:1785220581.643068:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510643968:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.643072:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510648064 00000100:00100000:3.0:1785220581.643076:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510648064 00000004:00080000:0.0:1785220581.643078:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64e00/2, rc 0, transno 4295070535 00000100:00100000:0.0:1785220581.643081:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510644608:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.643119:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284266 00000100:00100000:3.0:1785220581.643126:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1096:3654:x1871938510648064:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.643150:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510648320:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.643175:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.643278:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64000/2, rc 0, transno 4295070536 00000100:00100000:1.0:1785220581.643283:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510644480:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.643294:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64380/2, rc 0, transno 4295070534 00000100:00100000:1.0:1785220581.643298:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510643712:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.643303:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.643310:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510648448:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.643335:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.643383:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.643403:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.643452:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510648320 00000100:00100000:2.0:1785220581.643457:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510648320 00000100:00100000:2.0:1785220581.643480:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510648448 00000100:00100000:2.0:1785220581.643483:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510648448 00000100:00100000:2.0:1785220581.643501:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284267 00000100:00100000:2.0:1785220581.643507:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1098:3655:x1871938510648320:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.643521:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29185 count 1 00010000:00010000:2.0:1785220581.643534:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643546:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa35494e/0xf62499ab9278774a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b48d expref: 358 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643553:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7201:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.643557:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7201:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.643561:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643580:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000aa35494e/0xf62499ab9278774a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b48d expref: 358 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643591:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000aa35494e/0xf62499ab9278774a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b48d expref: 358 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.643602:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510648576:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.643638:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.643654:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284268 00000100:00100000:2.0:1785220581.643661:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1098:3653:x1871938510648448:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.643675:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29186 count 1 00010000:00010000:2.0:1785220581.643686:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643696:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d830fc18/0xf62499ab927877eb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4d3 expref: 358 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643702:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7202:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.643706:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7202:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.643709:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643725:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d830fc18/0xf62499ab927877eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4d3 expref: 358 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.643735:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d830fc18/0xf62499ab927877eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4d3 expref: 358 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.643744:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510648704:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.643779:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00002000:00080000:3.0:1785220581.643826:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29184 count 1 00010000:00010000:3.0:1785220581.643839:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.643848:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.643850:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ddaadff3/0xf62499ab9278768d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b41d expref: 358 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.643856:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7200:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.643859:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7200:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.643867:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.643888:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ddaadff3/0xf62499ab9278768d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b41d expref: 358 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.643900:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ddaadff3/0xf62499ab9278768d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b41d expref: 358 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.643910:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510648832:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.643934:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.646295:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510647040:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.646326:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.646367:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.646441:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931520 00000100:00100000:2.0:1785220581.646448:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931520 00010000:00010000:2.0:1785220581.646485:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003c899da1/0xf62499ab92789cff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7fe expref: 318 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.646514:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931392 00000100:00100000:2.0:1785220581.646518:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931392 00010000:00010000:2.0:1785220581.646532:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d903f05b/0xf62499ab92789c34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c795 expref: 319 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.646551:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931648 00000100:00100000:2.0:1785220581.646554:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931648 00010000:00010000:2.0:1785220581.646567:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003f5182a9/0xf62499ab92789ddf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c860 expref: 320 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.646587:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100190 00000100:00100000:2.0:1785220581.646597:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c307c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+320:2407:x1871938519931520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.646611:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.646616:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003c899da1/0xf62499ab92789cff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7fe expref: 320 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.646628:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003c899da1/0xf62499ab92789cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7fe expref: 320 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.646650:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c899da1/0xf62499ab92789cff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7fe expref: 320 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.646667:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.646671:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.646677:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.646685:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.646711:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.646755:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c307c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+319:2407:x1871938519931520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 159us (3372us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.646791:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100191 00000100:00100000:0.0:1785220581.646802:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+319:2407:x1871938519931392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.646811:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.646816:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d903f05b/0xf62499ab92789c34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c795 expref: 319 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646826:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d903f05b/0xf62499ab92789c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c795 expref: 319 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646845:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d903f05b/0xf62499ab92789c34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c795 expref: 319 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646856:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.646859:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.646862:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646869:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646889:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.646909:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+318:2407:x1871938519931392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 110us (3505us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.646916:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100191 00000100:00100000:0.0:1785220581.646936:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100192 00000100:00100000:0.0:1785220581.646943:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+317:2407:x1871938519931648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.646951:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.646956:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003f5182a9/0xf62499ab92789ddf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c860 expref: 317 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646965:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003f5182a9/0xf62499ab92789ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c860 expref: 317 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646982:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f5182a9/0xf62499ab92789ddf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c860 expref: 317 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.646992:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.646995:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x722f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.646999:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.647006:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.647336:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100190 00000100:00100000:2.0:1785220581.647404:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510646912:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.647419:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.647442:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.647450:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.647459:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.647467:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.647484:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.647649:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1162:3652:x1871938510645504:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22563us (22963us total) trans 4295070537 rc 0/0 00000100:00100000:2.0:1785220581.647657:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284257 00010000:00010000:0.0:1785220581.647966:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.647992:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+315:2407:x1871938519931648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1050us (4143us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.647999:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100192 00010000:00010000:2.0:1785220581.648460:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.648466:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648476:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4ce00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.648480:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.648484:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648489:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.648494:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.648501:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648505:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.648509:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648512:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeda96fd/0xf62499ab927a4b8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.648524:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.648530:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.648533:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648538:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.648541:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648548:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648561:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648567:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4b94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.648580:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.648582:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.648584:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.648620:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1162:3655:x1871938510645888:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23376us (23771us total) trans 4295070538 rc 0/0 00000100:00100000:0.0:1785220581.648627:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284258 00010000:00010000:0.0:1785220581.648646:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.648659:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510649216:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.648692:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.648736:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb65f80/2, rc 0, transno 4295070538 00000100:00100000:0.0:1785220581.648747:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510645888:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.648757:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.648774:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510649088:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.648798:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.649016:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510649216 00000100:00100000:2.0:1785220581.649023:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510649216 00000100:00100000:2.0:1785220581.649052:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510649088 00000100:00100000:2.0:1785220581.649055:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510649088 00000100:00100000:2.0:1785220581.649068:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284269 00000100:00100000:2.0:1785220581.649074:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1163:3655:x1871938510649216:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.649112:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29235 count 1 00010000:00010000:2.0:1785220581.649141:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649157:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003d89ff32/0xf62499ab9278a1ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca2e expref: 314 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649165:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7233:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.649169:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7233:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.649174:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649203:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003d89ff32/0xf62499ab9278a1ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca2e expref: 314 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649283:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003d89ff32/0xf62499ab9278a1ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca2e expref: 314 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.649295:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510649472:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.649334:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.649349:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284270 00000100:00100000:2.0:1785220581.649355:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1163:3655:x1871938510649088:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.649365:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29234 count 1 00010000:00010000:2.0:1785220581.649376:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649386:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd20c95d/0xf62499ab9278a09b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9a2 expref: 314 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649393:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7232:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.649395:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7232:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.649399:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649412:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fd20c95d/0xf62499ab9278a09b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9a2 expref: 314 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.649426:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fd20c95d/0xf62499ab9278a09b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9a2 expref: 314 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.649434:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510649600:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.649452:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.650142:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510648960:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.650178:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.650190:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.650201:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.650201:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510649728:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.650222:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.650242:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.650248:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510649344:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.650269:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.650282:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.650291:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.650307:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.650435:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1164:3655:x1871938510645760:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24966us (25539us total) trans 4295070539 rc 0/0 00000100:00100000:2.0:1785220581.650443:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284259 00000100:00100000:1.0:1785220581.650451:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.650465:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510648960 00000100:00100000:2.0:1785220581.650470:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510648960 00000100:00100000:2.0:1785220581.650493:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510649728 00000100:00100000:2.0:1785220581.650496:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510649728 00000100:00100000:2.0:1785220581.650514:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510649344 00000100:00100000:2.0:1785220581.650517:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510649344 00000100:00100000:2.0:1785220581.650535:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284271 00000100:00100000:2.0:1785220581.650541:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1166:3653:x1871938510648960:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.650557:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29233 count 1 00000100:00100000:1.0:1785220581.650591:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284272 00000100:00100000:1.0:1785220581.650600:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1166:3653:x1871938510649728:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.650614:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29237 count 1 00010000:00010000:1.0:1785220581.650629:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650643:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b2d9877b/0xf62499ab9278a36c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb00 expref: 314 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650650:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7235:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.650654:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7235:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.650658:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650683:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b2d9877b/0xf62499ab9278a36c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb00 expref: 314 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650693:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b2d9877b/0xf62499ab9278a36c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb00 expref: 314 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.650703:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0000_UUID:14258:1871938510649856:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.650744:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.650774:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284273 00000100:00100000:1.0:1785220581.650782:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1166:3653:x1871938510649344:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.650794:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29236 count 1 00010000:00010000:1.0:1785220581.650806:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650822:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd2121ed/0xf62499ab9278a2c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cab3 expref: 314 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650829:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7234:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.650832:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7234:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.650836:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650855:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dd2121ed/0xf62499ab9278a2c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cab3 expref: 314 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.650864:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dd2121ed/0xf62499ab9278a2c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cab3 expref: 314 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.650873:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510649984:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.650896:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.651023:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.651456:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651469:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000561cafe4/0xf62499ab9278a001 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c94e expref: 314 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651475:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7231:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.651478:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7231:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.651481:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651501:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000561cafe4/0xf62499ab9278a001 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c94e expref: 314 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651511:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000561cafe4/0xf62499ab9278a001 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c94e expref: 314 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.651521:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510650112:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.651558:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.651577:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.651583:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651593:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.651597:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651605:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651621:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651628:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4b9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.651643:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.651645:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.651647:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.651674:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510647296:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.651693:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.651862:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931776 00000100:00100000:0.0:1785220581.651868:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931776 00010000:00010000:0.0:1785220581.651896:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000045a297e1/0xf62499ab92787242 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b217 expref: 359 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.651935:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519931904 00000100:00100000:0.0:1785220581.651939:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519931904 00010000:00010000:0.0:1785220581.651953:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b7cee91/0xf62499ab927872ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b26b expref: 360 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.651980:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100193 00000100:00100000:0.0:1785220581.651988:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+360:2407:x1871938519931776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.651998:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.652004:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000045a297e1/0xf62499ab92787242 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b217 expref: 360 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652015:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000045a297e1/0xf62499ab92787242 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b217 expref: 360 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652031:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000045a297e1/0xf62499ab92787242 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b217 expref: 360 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652043:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.652047:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.652052:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652061:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652083:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.652125:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+359:2407:x1871938519931776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 137us (5758us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.652132:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100193 00000100:00100000:0.0:1785220581.652148:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100194 00000100:00100000:0.0:1785220581.652154:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+358:2407:x1871938519931904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.652163:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.652168:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b7cee91/0xf62499ab927872ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b26b expref: 358 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652178:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b7cee91/0xf62499ab927872ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b26b expref: 358 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652192:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b7cee91/0xf62499ab927872ce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b26b expref: 358 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652201:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.652204:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.652208:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652215:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.652236:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.652255:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+357:2407:x1871938519931904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (1804us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.652260:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100194 00000100:00100000:0.0:1785220581.652292:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510647552:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.652305:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.652344:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb66300/2, rc 0, transno 4295070539 00000100:00100000:0.0:1785220581.652354:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510645760:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.652533:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb67800/2, rc 0, transno 4295070537 00000100:00100000:3.0:1785220581.652538:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510645504:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.652659:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.652670:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.652680:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.652687:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.652702:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.652906:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.652914:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.652922:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.652926:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.652935:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.652948:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.652960:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ebc9a5d1/0xf62499ab927a4ba2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.652980:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.652982:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.652984:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.653240:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510650368:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.654081:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.654219:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.654229:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510650240:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.654260:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.654660:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510650368 00000100:00100000:2.0:1785220581.654667:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510650368 00000100:00100000:2.0:1785220581.654718:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510650240 00000100:00100000:2.0:1785220581.654722:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510650240 00000100:00100000:1.0:1785220581.654728:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932032 00000100:00100000:1.0:1785220581.654732:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932032 00000100:00100000:2.0:1785220581.654743:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284274 00000100:00100000:2.0:1785220581.654750:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1168:3655:x1871938510650368:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.654755:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ee45469d/0xf62499ab92789eaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8c2 expref: 315 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.654785:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29239 count 1 00000100:00100000:1.0:1785220581.654791:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100195 00000100:00100000:1.0:1785220581.654798:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb22d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+315:2407:x1871938519932032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.654806:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.654810:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.654815:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ee45469d/0xf62499ab92789eaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8c2 expref: 315 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654821:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000605848e6/0xf62499ab9278a64b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc11 expref: 315 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.654825:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ee45469d/0xf62499ab92789eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8c2 expref: 315 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654845:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7237:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.654845:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee45469d/0xf62499ab92789eaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8c2 expref: 315 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654849:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7237:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.654854:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.654859:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7230:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.654861:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7230:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.654866:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.654874:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654880:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000605848e6/0xf62499ab9278a64b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc11 expref: 314 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654892:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000605848e6/0xf62499ab9278a64b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc11 expref: 314 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.654893:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.654903:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510650496:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.654930:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb22d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+314:2407:x1871938519932032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 133us (3907us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.654937:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.654937:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100195 00000100:00100000:2.0:1785220581.654952:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284275 00000100:00100000:2.0:1785220581.654959:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9573c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1168:3655:x1871938510650240:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.654963:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.654970:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29238 count 1 00010000:00010000:1.0:1785220581.654972:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654979:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.654981:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.654989:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654991:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c8e81205/0xf62499ab9278a4a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb69 expref: 313 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.654998:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7236:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.655001:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7236:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.655005:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655005:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.655021:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c8e81205/0xf62499ab9278a4a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb69 expref: 313 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655030:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c8e81205/0xf62499ab9278a4a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb69 expref: 313 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.655038:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510650624:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.655060:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.655112:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.655173:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1100:3653:x1871938510646784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 29340us (29461us total) trans 4295069950 rc 0/0 00000100:00100000:1.0:1785220581.655180:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284261 00000100:00100000:2.0:1785220581.655184:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510647808:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.655202:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655203:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655212:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655220:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655227:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655238:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.655242:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.655322:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1169:3655:x1871938510647168:12345-0@lo:6:osp-syn-0-0.0 Request processed in 26742us (26809us total) trans 4295070540 rc 0/0 00000100:00100000:1.0:1785220581.655332:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284262 00000100:00100000:2.0:1785220581.655342:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510648576:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.655353:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655365:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.655371:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.655378:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.655378:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.655382:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655392:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.655393:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.655405:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.655406:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932160 00000100:00100000:2.0:1785220581.655409:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932160 00010000:00010000:1.0:1785220581.655412:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4ba9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655425:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000635ebaed/0xf62499ab927873d8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2db expref: 357 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655428:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.655431:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655437:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4ca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.655440:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.655442:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932288 00000100:00100000:2.0:1785220581.655444:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932288 00010000:00010000:2.0:1785220581.655453:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005c791710/0xf62499ab927874bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b34b expref: 358 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655454:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655464:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.655469:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932544 00000100:00100000:2.0:1785220581.655471:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932544 00010000:00010000:1.0:1785220581.655472:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f92f3013/0xf62499ab927a4bb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655480:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000aa35494e/0xf62499ab9278774a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b48d expref: 359 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655485:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.655487:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.655488:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.655493:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932416 00000100:00100000:2.0:1785220581.655496:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932416 00000100:00100000:1.0:1785220581.655499:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100196 00000100:00100000:1.0:1785220581.655506:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+360:2407:x1871938519932160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.655507:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005dbbc968/0xf62499ab9278753d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b398 expref: 360 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655515:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.655520:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000635ebaed/0xf62499ab927873d8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2db expref: 360 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.655523:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100197 00000100:00100000:2.0:1785220581.655528:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+360:2406:x1871938519932288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.655531:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000635ebaed/0xf62499ab927873d8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2db expref: 360 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655535:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.655538:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005c791710/0xf62499ab927874bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b34b expref: 360 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655544:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000635ebaed/0xf62499ab927873d8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b2db expref: 360 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655547:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005c791710/0xf62499ab927874bf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b34b expref: 360 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655560:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.655563:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.655565:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c791710/0xf62499ab927874bf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b34b expref: 359 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655568:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655573:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.655576:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.655576:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655580:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.655588:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.655591:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.655600:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.655612:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+358:2407:x1871938519932160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (500us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.655613:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+358:2406:x1871938519932288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.655619:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100196 00010000:00010000:0.0:1785220581.655706:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.655716:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.655725:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.655738:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.655751:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.655856:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1100:3654:x1871938510647424:12345-0@lo:6:osp-syn-1-0.0 Request processed in 26496us (26941us total) trans 4295069951 rc 0/0 00000100:00100000:0.0:1785220581.655863:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284263 00000100:00100000:0.0:1785220581.656043:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510648192:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.656057:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656072:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.656076:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656083:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.656101:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.656105:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1099:3653:x1871938510646656:12345-0@lo:6:osp-syn-1-0.0 Request processed in 30408us (30459us total) trans 4295069949 rc 0/0 00010000:00010000:0.0:1785220581.656110:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.656113:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284260 00010000:00010000:0.0:1785220581.656122:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656147:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4bb7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656161:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.656163:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.656165:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.656184:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100197 00000004:00080000:0.0:1785220581.656190:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991b800/2, rc 0, transno 4295069950 00000100:00100000:0.0:1785220581.656200:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510646784:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.656209:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991aa00/2, rc 0, transno 4295069949 00000100:00100000:0.0:1785220581.656212:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510646656:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.656222:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100198 00000100:00100000:0.0:1785220581.656228:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100199 00000100:00100000:2.0:1785220581.656229:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+356:2407:x1871938519932544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.656233:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb67480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+356:2407:x1871938519932416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.656238:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.656239:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.656245:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aa35494e/0xf62499ab9278774a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b48d expref: 356 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656245:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005dbbc968/0xf62499ab9278753d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b398 expref: 356 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656256:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005dbbc968/0xf62499ab9278753d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b398 expref: 356 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656257:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aa35494e/0xf62499ab9278774a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b48d expref: 356 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656270:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005dbbc968/0xf62499ab9278753d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b398 expref: 356 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656273:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa35494e/0xf62499ab9278774a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b48d expref: 356 pid: 14200 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656280:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.656281:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7201:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.656284:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7201:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.656284:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x71ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.656290:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656290:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656300:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656300:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656316:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.656316:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.656342:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb67480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2407:x1871938519932416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (947us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.656348:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100199 00010000:00010000:0.0:1785220581.656367:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656379:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656387:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656395:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.656405:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2407:x1871938519932544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 177us (1025us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.656408:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.656413:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100198 00010000:00010000:2.0:1785220581.656435:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656444:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656452:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656458:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656471:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.656487:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1099:3652:x1871938510647936:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13690us (13790us total) trans 4295069952 rc 0/0 00000100:00100000:0.0:1785220581.656494:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284265 00000100:00100000:2.0:1785220581.656550:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1099:3653:x1871938510647680:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21798us (22511us total) trans 4295069953 rc 0/0 00000100:00100000:2.0:1785220581.656558:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284264 00000004:00080000:0.0:1785220581.656561:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b109180/2, rc 0, transno 4295069953 00000100:00100000:0.0:1785220581.656565:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b109180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510647680:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.656569:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:2.0:1785220581.656574:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:0.0:1785220581.656578:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510650752:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.656579:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656585:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.656589:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656596:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.656606:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.656607:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656614:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4bc5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656627:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.656631:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.656631:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510650752 00000100:00100000:0.0:1785220581.656635:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510650752 00010000:00010000:2.0:1785220581.656642:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.656645:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656651:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.656658:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284276 00010000:00010000:2.0:1785220581.656660:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.656664:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1099:3653:x1871938510650752:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.656668:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d098f110/0xf62499ab927a4bbe lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x71fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656678:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.656680:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00080000:0.0:1785220581.656680:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29187 count 1 00010000:00010000:2.0:1785220581.656682:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.656688:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.656696:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656696:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656704:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656709:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000a268193/0xf62499ab9278784d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b512 expref: 352 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656712:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656717:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7203:0x0].0x0 since=29s v=(0 0) 00010000:00010000:0.0:1785220581.656720:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7203:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.656724:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656726:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.656738:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.656745:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000a268193/0xf62499ab9278784d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b512 expref: 352 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656756:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000a268193/0xf62499ab9278784d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b512 expref: 352 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.656787:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510650880:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.656822:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.656825:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1100:3655:x1871938510648320:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13318us (13648us total) trans 4295069954 rc 0/0 00010000:00010000:0.0:1785220581.656835:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.656839:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656846:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.656849:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656856:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656866:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656873:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a455e97/0xf62499ab927a4bcc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7201:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.656888:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.656889:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.656892:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.656921:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510651008:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.656943:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284267 00000100:00100000:0.0:1785220581.657018:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.657019:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510648704:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.657032:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.657075:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b108e00/2, rc 0, transno 4295069954 00000100:00100000:2.0:1785220581.657191:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.657250:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.657296:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510649472:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.657310:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.657398:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510648320:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.657411:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932672 00000100:00100000:0.0:1785220581.657414:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932672 00000100:00100000:2.0:1785220581.657419:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510649600:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.657432:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.657432:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d830fc18/0xf62499ab927877eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4d3 expref: 353 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.657453:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932800 00000100:00100000:0.0:1785220581.657456:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932800 00000100:00100000:2.0:1785220581.657459:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.657467:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ddaadff3/0xf62499ab9278768d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b41d expref: 354 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.657476:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.657482:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933056 00000100:00100000:0.0:1785220581.657485:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933056 00000100:00100000:2.0:1785220581.657487:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519932928 00000100:00100000:2.0:1785220581.657489:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519932928 00010000:00010000:0.0:1785220581.657496:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fd20c95d/0xf62499ab9278a09b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9a2 expref: 315 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.657500:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003d89ff32/0xf62499ab9278a1ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca2e expref: 315 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.657515:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100200 00000100:00100000:0.0:1785220581.657522:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2407:x1871938519932672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.657565:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100201 00000100:00100000:1.0:1785220581.657666:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100202 00000100:00100000:1.0:1785220581.657675:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+315:2404:x1871938519933056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.657691:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.657697:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fd20c95d/0xf62499ab9278a09b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9a2 expref: 315 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657708:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fd20c95d/0xf62499ab9278a09b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9a2 expref: 315 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657723:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd20c95d/0xf62499ab9278a09b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9a2 expref: 315 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657739:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7232:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.657742:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7232:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.657745:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100203 00010000:00010000:1.0:1785220581.657746:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657754:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.657755:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+314:2404:x1871938519932928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.657774:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.657778:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003d89ff32/0xf62499ab9278a1ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca2e expref: 314 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657782:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.657788:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003d89ff32/0xf62499ab9278a1ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca2e expref: 314 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.657802:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+314:2404:x1871938519933056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 129us (343us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.657804:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d89ff32/0xf62499ab9278a1ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca2e expref: 314 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.657808:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100202 00010000:00010000:3.0:1785220581.657815:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7233:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.657817:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7233:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.657821:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657825:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.657832:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2407:x1871938519932800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.657841:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.657845:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ddaadff3/0xf62499ab9278768d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b41d expref: 354 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657855:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ddaadff3/0xf62499ab9278768d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b41d expref: 354 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657867:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7200:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.657871:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7200:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.657875:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657883:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.657885:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657896:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.657903:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.657913:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2407:x1871938519932800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 342us (663us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.657917:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+312:2404:x1871938519932928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 167us (441us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.657920:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100201 00000100:00100000:3.0:1785220581.657923:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100203 00010000:00010000:3.0:1785220581.657949:0:14183:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddaadff3/0xf62499ab9278768d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b41d expref: 353 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.657960:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510648832:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.657970:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0000_UUID:14258:1871938510649856:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.657972:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.657980:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.657983:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.657993:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658002:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658009:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658023:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.658058:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510649984:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.658069:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.658140:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1100:3654:x1871938510648064:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15013us (15159us total) trans 4295069955 rc 0/0 00000100:00100000:3.0:1785220581.658147:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284266 00000100:00100000:3.0:1785220581.658162:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510651008 00000100:00100000:3.0:1785220581.658165:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510651008 00000100:00100000:3.0:1785220581.658186:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284277 00000100:00100000:3.0:1785220581.658192:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1100:3653:x1871938510651008:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.658207:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29188 count 1 00010000:00010000:3.0:1785220581.658222:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.658224:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.658230:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d830fc18/0xf62499ab927877eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4d3 expref: 352 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658235:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000117511ac/0xf62499ab927878af lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b558 expref: 352 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.658239:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d830fc18/0xf62499ab927877eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4d3 expref: 352 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658242:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7204:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.658246:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7204:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.658250:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.658252:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d830fc18/0xf62499ab927877eb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b4d3 expref: 352 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.658262:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7202:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.658265:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7202:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.658268:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658271:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000117511ac/0xf62499ab927878af lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b558 expref: 351 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.658277:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658281:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000117511ac/0xf62499ab927878af lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b558 expref: 351 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.658292:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.658293:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510651264:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.658357:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+351:2407:x1871938519932672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 800us (1132us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.658364:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100200 00000004:00080000:0.0:1785220581.658427:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601d880/2, rc 0, transno 4295070540 00000100:00100000:0.0:1785220581.658438:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510647168:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.658450:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510651392:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.658467:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.658474:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.658488:0:14260:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510651392 00000100:00100000:3.0:1785220581.658492:0:14260:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510651392 00000100:00100000:0.0:1785220581.658494:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.658500:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510651136:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.658515:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284278 00000100:00100000:0.0:1785220581.658519:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.658520:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1101:3655:x1871938510651392:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.658532:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29190 count 1 00010000:00010000:3.0:1785220581.658543:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.658550:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b109c00/2, rc 0, transno 4295069955 00010000:00010000:3.0:1785220581.658555:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000037ae2efc/0xf62499ab92787a8b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b669 expref: 350 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.658557:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b109c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510648064:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.658562:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7206:0x0].0x0 since=29s v=(0 0) 00010000:00010000:3.0:1785220581.658565:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7206:0x0].0x0 since=0s v=(1 0) 00000004:00080000:0.0:1785220581.658567:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b109880/2, rc 0, transno 4295069951 00010000:00010000:3.0:1785220581.658569:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.658570:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b109880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510647424:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.658587:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000037ae2efc/0xf62499ab92787a8b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b669 expref: 350 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.658597:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000037ae2efc/0xf62499ab92787a8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b669 expref: 350 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.658603:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.658606:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510651520:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.658609:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510651648:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.658693:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510651136 00000100:00100000:2.0:1785220581.658698:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510651136 00000100:00100000:2.0:1785220581.658722:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284279 00000100:00100000:2.0:1785220581.658728:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1102:3655:x1871938510651136:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.658740:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29189 count 1 00010000:00010000:2.0:1785220581.658751:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.658754:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.658773:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009cf08519/0xf62499ab927879ce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5f2 expref: 350 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.658780:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7205:0x0].0x0 since=29s v=(0 0) 00010000:00010000:2.0:1785220581.658783:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7205:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.658787:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.658804:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009cf08519/0xf62499ab927879ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5f2 expref: 350 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.658808:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510651776:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.658814:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009cf08519/0xf62499ab927879ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5f2 expref: 350 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.658828:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.658898:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.658913:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510651648 00000100:00100000:3.0:1785220581.658917:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510651648 00000100:00100000:3.0:1785220581.659225:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510651776 00000100:00100000:3.0:1785220581.659229:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510651776 00000100:00100000:3.0:1785220581.659252:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284280 00000100:00100000:3.0:1785220581.659259:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1104:3653:x1871938510651648:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.659272:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29191 count 1 00010000:00010000:3.0:1785220581.659284:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.659296:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d6928d/0xf62499ab92787bf7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6d9 expref: 350 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.659298:0:14169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284281 00010000:00010000:3.0:1785220581.659304:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7207:0x0].0x0 since=28s v=(0 0) 00000100:00100000:0.0:1785220581.659306:0:14169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1104:3653:x1871938510651776:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.659307:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7207:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.659312:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220581.659318:0:14169:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29192 count 1 00010000:00010000:0.0:1785220581.659329:0:14169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.659335:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a4d6928d/0xf62499ab92787bf7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6d9 expref: 350 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.659340:0:14169:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7e52555/0xf62499ab92787d78 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b765 expref: 350 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.659344:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a4d6928d/0xf62499ab92787bf7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6d9 expref: 350 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.659347:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7208:0x0].0x0 since=28s v=(0 0) 00010000:00010000:0.0:1785220581.659350:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7208:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.659354:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510652032:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.659354:0:14169:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.659369:0:14169:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b7e52555/0xf62499ab92787d78 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b765 expref: 350 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.659378:0:14169:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b7e52555/0xf62499ab92787d78 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b765 expref: 350 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.659379:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.659387:0:14169:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510652160:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.659408:0:14169:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26540 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.659427:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10a680/2, rc 0, transno 4295069952 00000100:00100000:3.0:1785220581.659583:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510647936:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.659853:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.659862:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510652288:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.659874:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510650112:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.659888:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.659890:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.659908:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510652288 00000100:00100000:0.0:1785220581.659911:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510652288 00000100:00100000:0.0:1785220581.659937:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284282 00000100:00100000:0.0:1785220581.659944:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1105:3655:x1871938510652288:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.659957:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29193 count 1 00000100:00100000:2.0:1785220581.659971:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510651904:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.659971:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.659983:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002d72efa6/0xf62499ab92787e35 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b7d5 expref: 350 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.659990:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7209:0x0].0x0 since=28s v=(0 0) 00010000:00010000:0.0:1785220581.659993:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7209:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.659997:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.660007:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.660016:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002d72efa6/0xf62499ab92787e35 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b7d5 expref: 350 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660025:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002d72efa6/0xf62499ab92787e35 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b7d5 expref: 350 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660031:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660040:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.660042:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510652416:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.660049:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660057:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.660062:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.660081:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220581.660194:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660203:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660210:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660217:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660230:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.660249:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1169:3655:x1871938510649088:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10893us (11449us total) trans 4295070541 rc 0/0 00000100:00100000:2.0:1785220581.660256:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284270 00010000:00010000:2.0:1785220581.660279:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660288:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660302:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.660305:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1169:3655:x1871938510649216:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11231us (11613us total) trans 4295070542 rc 0/0 00010000:00010000:2.0:1785220581.660309:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.660312:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284269 00010000:00010000:2.0:1785220581.660320:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:0.0:1785220581.660329:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220581.660335:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660343:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.660346:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660356:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660374:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660381:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a4bda lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7200:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.660395:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1106:3653:x1871938510648448:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16735us (17060us total) trans 4295069956 rc 0/0 00010000:00010000:0.0:1785220581.660399:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00000100:00100000:2.0:1785220581.660401:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284268 00010000:00010000:0.0:1785220581.660403:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660409:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4da00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.660412:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660416:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.660419:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660421:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660428:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.660430:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660431:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660438:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecdc5d08/0xf62499ab927a4be8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660439:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660450:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660456:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a4be1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660469:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.660470:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.660471:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.660472:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.660473:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.660478:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4de00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.660481:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660488:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.660495:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c681c00/2, rc 0, transno 4295069956 00010000:00010000:2.0:1785220581.660498:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.660502:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510648448:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.660505:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a4bd3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7202:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660519:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.660522:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.660523:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.660537:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510652544:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.660557:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.660560:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.660565:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510652672:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.660581:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601ce00/2, rc 0, transno 4295070541 00000100:00100000:2.0:1785220581.660586:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.660589:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510649088:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.660599:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601e300/2, rc 0, transno 4295070542 00000100:00100000:0.0:1785220581.660602:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510649216:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.660637:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510652800:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.660666:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.660700:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.660706:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510652928:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.660727:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.660747:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510652544 00000100:00100000:2.0:1785220581.660751:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510652544 00000100:00100000:2.0:1785220581.660788:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510652672 00000100:00100000:2.0:1785220581.660791:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510652672 00000100:00100000:2.0:1785220581.660806:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510652800 00000100:00100000:2.0:1785220581.660809:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510652800 00000100:00100000:2.0:1785220581.660823:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510652928 00000100:00100000:2.0:1785220581.660826:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510652928 00000100:00100000:2.0:1785220581.660842:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284283 00000100:00100000:2.0:1785220581.660849:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1106:3655:x1871938510652544:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.660862:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29194 count 1 00010000:00010000:2.0:1785220581.660876:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660889:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008c0831a0/0xf62499ab92787ee4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b845 expref: 350 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660896:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x720a:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.660900:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.660904:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660922:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008c0831a0/0xf62499ab92787ee4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b845 expref: 350 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.660931:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008c0831a0/0xf62499ab92787ee4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b845 expref: 350 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.660942:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510653056:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.660970:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284284 00000100:00100000:2.0:1785220581.660972:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.660980:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1171:3653:x1871938510652672:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.660992:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29240 count 1 00000100:00100000:2.0:1785220581.660999:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284285 00010000:00010000:1.0:1785220581.661004:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.661005:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1171:3653:x1871938510652800:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.661015:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29241 count 1 00010000:00010000:1.0:1785220581.661016:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000756a31d1/0xf62499ab9278a724 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc88 expref: 311 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661024:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.661028:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7238:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.661031:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7238:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.661034:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661035:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000056dd9e53/0xf62499ab9278a87b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd0d expref: 311 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661042:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7239:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.661045:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7239:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.661049:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.661053:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000756a31d1/0xf62499ab9278a724 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc88 expref: 311 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.661063:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000756a31d1/0xf62499ab9278a724 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc88 expref: 311 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661064:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000056dd9e53/0xf62499ab9278a87b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd0d expref: 311 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661073:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000056dd9e53/0xf62499ab9278a87b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd0d expref: 311 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.661074:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510653184:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.661082:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955ff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510653312:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.661129:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.661144:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.661160:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284286 00000100:00100000:2.0:1785220581.661167:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1171:3655:x1871938510652928:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.661179:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29242 count 1 00010000:00010000:2.0:1785220581.661190:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661201:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e592779e/0xf62499ab9278a94d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cda0 expref: 311 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661208:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x723a:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.661210:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.661211:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.661214:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661231:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e592779e/0xf62499ab9278a94d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cda0 expref: 311 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.661240:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e592779e/0xf62499ab9278a94d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cda0 expref: 311 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.661249:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510653440:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.661273:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.662155:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.662727:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.662785:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933312 00000100:00100000:2.0:1785220581.662796:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933312 00010000:00010000:2.0:1785220581.662913:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000561cafe4/0xf62499ab9278a001 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c94e expref: 312 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.662986:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100204 00000100:00100000:2.0:1785220581.663008:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c304050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+312:2407:x1871938519933312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.663036:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.663045:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000561cafe4/0xf62499ab9278a001 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c94e expref: 312 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663073:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000561cafe4/0xf62499ab9278a001 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c94e expref: 312 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663407:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000561cafe4/0xf62499ab9278a001 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c94e expref: 312 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663422:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7231:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.663426:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7231:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.663432:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663443:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663472:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.663524:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c304050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+311:2407:x1871938519933312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 518us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.663532:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100204 00000100:00100000:0.0:1785220581.663546:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.663575:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933440 00000100:00100000:2.0:1785220581.663579:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933440 00010000:00010000:2.0:1785220581.663594:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dd2121ed/0xf62499ab9278a2c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cab3 expref: 311 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.663615:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100205 00000100:00100000:0.0:1785220581.663616:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.663621:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+311:2405:x1871938519933440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.663629:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.663633:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dd2121ed/0xf62499ab9278a2c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cab3 expref: 311 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663643:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dd2121ed/0xf62499ab9278a2c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cab3 expref: 311 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663655:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd2121ed/0xf62499ab9278a2c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cab3 expref: 311 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.663665:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7234:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.663669:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7234:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.663672:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.663675:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933184 00000100:00100000:1.0:1785220581.663679:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933184 00010000:00010000:2.0:1785220581.663680:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663697:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d9877b/0xf62499ab9278a36c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb00 expref: 311 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.663719:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100206 00000100:00100000:1.0:1785220581.663726:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+311:2407:x1871938519933184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.663734:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.663739:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b2d9877b/0xf62499ab9278a36c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb00 expref: 311 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663754:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b2d9877b/0xf62499ab9278a36c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb00 expref: 311 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663777:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2d9877b/0xf62499ab9278a36c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb00 expref: 311 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663789:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7235:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.663792:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7235:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.663796:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663804:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663823:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.663846:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+310:2407:x1871938519933184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 120us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.663853:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100206 00010000:00010000:3.0:1785220581.663871:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663873:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663883:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.663884:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.663894:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663894:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663903:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.663908:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.663917:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.663921:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.664078:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1173:3653:x1871938510648960:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13535us (13897us total) trans 4295070543 rc 0/0 00000100:00100000:3.0:1785220581.664119:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284271 00000100:00100000:1.0:1785220581.664298:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1172:3653:x1871938510649728:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13699us (14073us total) trans 4295070544 rc 0/0 00000100:00100000:1.0:1785220581.664305:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284272 00010000:00010000:2.0:1785220581.664343:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.664366:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+309:2405:x1871938519933440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 744us (818us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.664373:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100205 00010000:00010000:2.0:1785220581.664390:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.664396:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664405:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.664409:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664417:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664430:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664437:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4bef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664455:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.664455:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.664457:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.664459:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.664460:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.664467:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.664470:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.664476:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.664491:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.664494:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601d500/2, rc 0, transno 4295070543 00010000:00010000:0.0:1785220581.664498:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a4bfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.664503:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510648960:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.664512:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601c380/2, rc 0, transno 4295070544 00010000:00010000:0.0:1785220581.664512:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.664513:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.664515:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510649728:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.664515:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.664525:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.664533:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.664541:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.664549:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.664566:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.664650:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1172:3653:x1871938510649344:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13868us (14379us total) trans 4295070545 rc 0/0 00000100:00100000:0.0:1785220581.664657:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284273 00000004:00080000:2.0:1785220581.664667:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601d180/2, rc 0, transno 4295070545 00000100:00100000:2.0:1785220581.664671:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510649344:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.664681:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.664686:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664693:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.664696:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664704:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664715:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664722:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a4bf6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.664735:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.664737:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.664739:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.665179:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.665202:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510653568:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.665255:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.665316:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510653824:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.665344:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.665374:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.665379:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510653696:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.665394:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510653568 00000100:00100000:2.0:1785220581.665399:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.665399:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510653568 00000100:00100000:0.0:1785220581.665430:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510653824 00000100:00100000:0.0:1785220581.665434:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510653824 00000100:00100000:2.0:1785220581.665436:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510653696 00000100:00100000:2.0:1785220581.665439:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510653696 00000100:00100000:0.0:1785220581.665455:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284287 00000100:00100000:2.0:1785220581.665459:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284288 00000100:00100000:0.0:1785220581.665461:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1174:3654:x1871938510653568:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.665464:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1174:3653:x1871938510653824:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.665481:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29245 count 1 00002000:00080000:0.0:1785220581.665481:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29243 count 1 00010000:00010000:2.0:1785220581.665499:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.665499:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.665516:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000070c16d85/0xf62499ab9278ab84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cebf expref: 308 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.665516:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e512a538/0xf62499ab9278a9ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cded expref: 308 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.665525:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x723d:0x0].0x0 since=26s v=(0 0) 00010000:00010000:0.0:1785220581.665525:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x723b:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.665529:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.665529:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.665534:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.665535:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.665556:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000070c16d85/0xf62499ab9278ab84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cebf expref: 308 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.665556:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e512a538/0xf62499ab9278a9ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cded expref: 308 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.665570:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000070c16d85/0xf62499ab9278ab84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cebf expref: 308 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.665570:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e512a538/0xf62499ab9278a9ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cded expref: 308 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.665583:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510653952:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.665583:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510654080:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.665625:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.665629:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284289 00000100:00100000:1.0:1785220581.665638:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1174:3653:x1871938510653696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.665649:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29244 count 1 00010000:00010000:1.0:1785220581.665661:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.665673:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cae27715/0xf62499ab9278aace lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce56 expref: 308 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.665680:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x723c:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.665682:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.665686:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.665704:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cae27715/0xf62499ab9278aace lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce56 expref: 308 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.665714:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cae27715/0xf62499ab9278aace lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce56 expref: 308 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.665723:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0000_UUID:14266:1871938510654208:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.665779:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.667218:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.670272:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.670293:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.670318:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933568 00000100:00100000:0.0:1785220581.670327:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933568 00010000:00010000:0.0:1785220581.670367:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000605848e6/0xf62499ab9278a64b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc11 expref: 309 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.670399:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933824 00000100:00100000:0.0:1785220581.670403:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933824 00010000:00010000:0.0:1785220581.670419:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000a268193/0xf62499ab9278784d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b512 expref: 351 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.670442:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100207 00000100:00100000:0.0:1785220581.670451:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+309:2405:x1871938519933568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.670465:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.670471:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000605848e6/0xf62499ab9278a64b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc11 expref: 309 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.670482:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000605848e6/0xf62499ab9278a64b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc11 expref: 309 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.670499:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510650624:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.670505:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7237:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.670509:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7237:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.670515:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670521:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.670524:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.670544:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100208 00010000:00010000:0.0:1785220581.670547:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.670552:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+351:2404:x1871938519933824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.670560:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.670564:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000a268193/0xf62499ab9278784d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b512 expref: 351 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670573:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000a268193/0xf62499ab9278784d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b512 expref: 351 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670586:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7203:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.670589:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7203:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.670589:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+309:2405:x1871938519933568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 139us (327us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.670593:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.670596:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100207 00010000:00010000:3.0:1785220581.670602:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670613:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.670632:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+351:2404:x1871938519933824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (337us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.670638:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100208 00000100:00100000:0.0:1785220581.670639:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.670651:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933696 00000100:00100000:3.0:1785220581.670654:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933696 00010000:00010000:3.0:1785220581.670667:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c8e81205/0xf62499ab9278a4a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb69 expref: 309 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.670686:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100209 00000100:00100000:3.0:1785220581.670692:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+309:2405:x1871938519933696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.670699:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.670703:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c8e81205/0xf62499ab9278a4a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb69 expref: 309 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670712:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c8e81205/0xf62499ab9278a4a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb69 expref: 309 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670724:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c8e81205/0xf62499ab9278a4a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb69 expref: 309 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670743:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7236:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.670746:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7236:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.670750:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670757:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670784:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.670800:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.670802:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+308:2405:x1871938519933696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.670808:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100209 00000100:00100000:3.0:1785220581.670822:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519933952 00000100:00100000:3.0:1785220581.670825:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519933952 00000100:00100000:0.0:1785220581.670831:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.670844:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000117511ac/0xf62499ab927878af lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b558 expref: 351 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.670860:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934080 00000100:00100000:3.0:1785220581.670862:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934080 00000100:00100000:0.0:1785220581.670867:0:14169:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510652160:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.670874:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000037ae2efc/0xf62499ab92787a8b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b669 expref: 352 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.670880:0:14169:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.670890:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100210 00000100:00100000:3.0:1785220581.670897:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+352:2405:x1871938519933952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.670905:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.670910:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000117511ac/0xf62499ab927878af lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b558 expref: 352 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.670919:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.670920:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000117511ac/0xf62499ab927878af lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b558 expref: 352 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670932:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7204:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.670934:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.670935:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7204:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.670939:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.670947:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.670947:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.670958:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.670974:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+352:2405:x1871938519933952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (175us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.670978:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510652416:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.670981:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100210 00010000:00010000:0.0:1785220581.670990:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.670992:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934208 00000100:00100000:3.0:1785220581.670994:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934208 00010000:00010000:3.0:1785220581.671006:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d6928d/0xf62499ab92787bf7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6d9 expref: 352 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.671021:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934336 00000100:00100000:3.0:1785220581.671023:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934336 00010000:00010000:3.0:1785220581.671034:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b7e52555/0xf62499ab92787d78 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b765 expref: 353 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.671037:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.671051:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.671133:0:13908:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a268193/0xf62499ab9278784d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b512 expref: 353 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.671145:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510650880:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.671157:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671165:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671172:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671182:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671190:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671213:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.671262:0:14177:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000605848e6/0xf62499ab9278a64b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc11 expref: 307 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.671273:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510650496:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.671284:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.671292:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.671299:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.671306:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.671313:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.671326:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.671365:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1107:3653:x1871938510650752:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14700us (14757us total) trans 4295069957 rc 0/0 00000100:00100000:1.0:1785220581.671374:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284276 00010000:00010000:1.0:1785220581.671393:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.671400:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671408:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.671412:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.671415:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1175:3655:x1871938510650368:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16665us (17338us total) trans 4295070546 rc 0/0 00010000:00010000:1.0:1785220581.671421:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.671422:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284274 00010000:00010000:1.0:1785220581.671434:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671441:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a4c12 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7203:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671464:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.671468:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671478:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.671481:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671487:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671496:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671502:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000820fd93c/0xf62499ab927a4c04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.671514:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.671515:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.671517:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.671807:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934464 00000100:00100000:3.0:1785220581.671812:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934464 00010000:00010000:3.0:1785220581.672167:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009cf08519/0xf62499ab927879ce lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5f2 expref: 353 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.672197:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934592 00000100:00100000:0.0:1785220581.672220:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934720 00000100:00100000:0.0:1785220581.672224:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934720 00000100:00100000:2.0:1785220581.672240:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100211 00010000:00010000:0.0:1785220581.672244:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008c0831a0/0xf62499ab92787ee4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b845 expref: 354 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.672249:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9af450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2405:x1871938519934080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.672259:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.672265:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000037ae2efc/0xf62499ab92787a8b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b669 expref: 354 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.672270:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100212 00000100:00100000:0.0:1785220581.672276:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2405:x1871938519934208:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.672277:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000037ae2efc/0xf62499ab92787a8b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b669 expref: 354 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.672283:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.672287:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a4d6928d/0xf62499ab92787bf7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6d9 expref: 354 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.672292:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7206:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.672295:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a4d6928d/0xf62499ab92787bf7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6d9 expref: 354 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.672296:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7206:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.672302:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.672306:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7207:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.672309:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7207:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.672310:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.672313:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.672320:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.672326:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.672329:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.672355:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934592 00010000:00010000:3.0:1785220581.672375:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002d72efa6/0xf62499ab92787e35 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b7d5 expref: 355 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.672391:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100213 00000100:00100000:3.0:1785220581.672399:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+355:2405:x1871938519934336:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.672405:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.672409:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b7e52555/0xf62499ab92787d78 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b765 expref: 355 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.672417:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b7e52555/0xf62499ab92787d78 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b765 expref: 355 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.672430:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7e52555/0xf62499ab92787d78 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b765 expref: 355 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.672443:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7208:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.672445:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7208:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.672450:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.672458:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.672474:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.672581:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9af450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+354:2405:x1871938519934080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 332us (1748us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.672590:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100211 00000100:00100000:2.0:1785220581.672637:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100214 00000100:00100000:2.0:1785220581.672644:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2404:x1871938519934464:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.672651:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.672656:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009cf08519/0xf62499ab927879ce lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5f2 expref: 353 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.672666:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009cf08519/0xf62499ab927879ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5f2 expref: 353 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.672684:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7205:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.672688:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7205:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.672692:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.672700:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.672712:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.672741:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100215 00000100:00100000:1.0:1785220581.672751:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2405:x1871938519934720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.672768:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.672773:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008c0831a0/0xf62499ab92787ee4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b845 expref: 353 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.672782:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008c0831a0/0xf62499ab92787ee4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b845 expref: 353 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.672800:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.672803:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.672807:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.672814:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.672827:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.673021:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+353:2405:x1871938519934336:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 622us (2085us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.673028:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100213 00000100:00100000:3.0:1785220581.673043:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100216 00000100:00100000:3.0:1785220581.673051:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+352:2405:x1871938519934592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.673059:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.673064:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002d72efa6/0xf62499ab92787e35 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b7d5 expref: 352 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673074:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002d72efa6/0xf62499ab92787e35 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b7d5 expref: 352 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673113:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d72efa6/0xf62499ab92787e35 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b7d5 expref: 352 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673125:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7209:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.673128:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7209:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.673132:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673144:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673165:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.673184:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+351:2405:x1871938519934592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 135us (2147us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.673191:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100216 00000100:00100000:0.0:1785220581.673209:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e06d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+350:2405:x1871938519934208:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 932us (2288us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.673216:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100212 00010000:00010000:0.0:1785220581.673235:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673236:0:14183:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000117511ac/0xf62499ab927878af lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b558 expref: 349 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.673246:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510651264:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.673249:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.673258:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673263:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.673266:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673271:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673277:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.673281:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.673284:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673291:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673306:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.673418:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1108:3655:x1871938510652288:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13473us (13527us total) trans 4295069958 rc 0/0 00000100:00100000:3.0:1785220581.673419:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1108:3653:x1871938510651008:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15225us (16399us total) trans 4295069959 rc 0/0 00000100:00100000:3.0:1785220581.673427:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284277 00010000:00010000:3.0:1785220581.673460:0:14189:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d6928d/0xf62499ab92787bf7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6d9 expref: 348 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.673471:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510652032:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.673484:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673492:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673498:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673505:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673513:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.673527:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.673597:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1108:3653:x1871938510651648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14339us (14842us total) trans 4295069960 rc 0/0 00000100:00100000:2.0:1785220581.673669:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+347:2404:x1871938519934464:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1025us (2719us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.673677:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100214 00010000:00010000:2.0:1785220581.673711:0:14197:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cf08519/0xf62499ab927879ce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b5f2 expref: 346 pid: 14198 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.673723:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510651904:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.673736:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.673744:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.673751:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.673775:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.673783:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.673792:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:2.0:1785220581.674251:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1109:3655:x1871938510651136:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15522us (15729us total) trans 4295069961 rc 0/0 00000100:00100000:1.0:1785220581.674736:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+345:2405:x1871938519934720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1985us (3680us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.674748:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100215 00010000:00010000:1.0:1785220581.675001:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.675043:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675339:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:0.0:1785220581.675374:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284282 00010000:00010000:1.0:1785220581.675380:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.675392:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284280 00010000:00010000:0.0:1785220581.675433:0:14169:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.675441:0:14169:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.675450:0:14169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.675458:0:14169:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675471:0:14260:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000037ae2efc/0xf62499ab92787a8b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b669 expref: 344 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.675473:0:14169:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220581.675482:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510651520:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.675502:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675510:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675517:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675524:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675531:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675539:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1785220581.675606:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284279 00010000:00010000:1.0:1785220581.675612:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675628:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675640:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4c3c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7209:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.675642:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c0831a0/0xf62499ab92787ee4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b845 expref: 343 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.675652:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510653056:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.675657:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.675661:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.675665:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675668:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.675671:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.675673:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675679:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.675680:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.675689:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675689:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.675697:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675697:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4c19 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7204:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.675706:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00010000:00010000:1.0:1785220581.675716:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.675720:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675727:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.675730:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675738:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675747:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675754:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4c2e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7207:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675781:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.675785:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.675790:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1109:3655:x1871938510651392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17269us (17315us total) trans 4295069963 rc 0/0 00010000:00010000:1.0:1785220581.675792:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.675795:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.675797:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284278 00010000:00010000:1.0:1785220581.675802:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675812:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675818:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675819:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4c27 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7205:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675827:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675835:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675842:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.675851:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220581.675903:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.675908:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675914:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.675917:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675924:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.675933:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9573c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1175:3655:x1871938510650240:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20975us (21673us total) trans 4295070547 rc 0/0 00010000:00010000:1.0:1785220581.675935:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.675940:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284275 00010000:00010000:1.0:1785220581.675942:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a4c35 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7208:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675954:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.675957:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675963:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.675966:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675976:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675984:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.675990:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4c20 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7206:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676001:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.676003:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676009:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.676012:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676018:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676027:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676034:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4c43 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676049:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.676052:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676059:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4dc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.676062:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676070:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676244:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676254:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a4c0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676277:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.676279:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.676280:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.676376:0:14169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e04a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1108:3653:x1871938510651776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17071us (17545us total) trans 4295069962 rc 0/0 00000100:00100000:0.0:1785220581.676383:0:14169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284281 00000100:00100000:3.0:1785220581.676406:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.676408:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.676421:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934848 00000100:00100000:0.0:1785220581.676427:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934848 00010000:00010000:0.0:1785220581.676453:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000756a31d1/0xf62499ab9278a724 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc88 expref: 307 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.676478:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100217 00000100:00100000:0.0:1785220581.676486:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+307:2405:x1871938519934848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.676499:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.676504:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000756a31d1/0xf62499ab9278a724 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc88 expref: 307 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.676515:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000756a31d1/0xf62499ab9278a724 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc88 expref: 307 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.676534:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7238:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.676538:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7238:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.676543:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.676552:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.676567:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.676593:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+307:2405:x1871938519934848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.676599:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100217 00000100:00100000:0.0:1785220581.676631:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510654080:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.676647:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.676743:0:13818:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000756a31d1/0xf62499ab9278a724 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc88 expref: 306 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.676755:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510653184:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.676807:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1107:3655:x1871938510652544:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15958us (16245us total) trans 4295069964 rc 0/0 00000100:00100000:2.0:1785220581.676814:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284283 00010000:00010000:2.0:1785220581.676891:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.676910:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510653440:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.676923:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.676945:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955ff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510653312:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.676955:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.676976:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510653952:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.676986:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.677017:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e07100/2, rc 0, transno 4295069962 00010000:00010000:1.0:1785220581.677020:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677028:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677036:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677045:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677052:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677065:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.677303:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1175:3653:x1871938510652672:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16325us (16715us total) trans 4295070548 rc 0/0 00000100:00100000:1.0:1785220581.677310:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284284 00010000:00010000:1.0:1785220581.677326:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.677331:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677338:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.677341:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677349:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677362:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677369:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4c4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.677383:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.677385:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.677387:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.677519:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510651776:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.677536:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f727b80/2, rc 0, transno 4295069960 00000100:00100000:2.0:1785220581.677540:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510651648:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.677548:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b108a80/2, rc 0, transno 4295069959 00000100:00100000:2.0:1785220581.677551:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510651008:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.677559:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba4000/2, rc 0, transno 4295069957 00000100:00100000:2.0:1785220581.677561:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510650752:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.677574:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601ed80/2, rc 0, transno 4295070548 00000100:00100000:2.0:1785220581.677587:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510652672:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.677600:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.677628:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510654336:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.677661:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.677755:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510654720:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.677755:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510654848:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.677789:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.677807:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510654592:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.677810:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.677827:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.677835:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.677841:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510654464:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.677947:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.677967:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519934976 00000100:00100000:2.0:1785220581.677972:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519934976 00010000:00010000:2.0:1785220581.677992:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e592779e/0xf62499ab9278a94d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cda0 expref: 306 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.678015:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100218 00000100:00100000:2.0:1785220581.678024:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90700850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+306:2405:x1871938519934976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.678034:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.678040:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e592779e/0xf62499ab9278a94d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cda0 expref: 306 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678051:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e592779e/0xf62499ab9278a94d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cda0 expref: 306 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678071:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e592779e/0xf62499ab9278a94d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cda0 expref: 306 pid: 8752 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678083:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.678176:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.678182:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678192:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678213:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.678240:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90700850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+305:2405:x1871938519934976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 217us (294us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.678247:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100218 00010000:00010000:2.0:1785220581.678289:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678298:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678307:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678314:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678334:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.678444:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1175:3655:x1871938510652928:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17277us (17715us total) trans 4295070549 rc 0/0 00000100:00100000:2.0:1785220581.678451:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284286 00000100:00100000:2.0:1785220581.678465:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510654336 00000100:00100000:2.0:1785220581.678468:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510654336 00000100:00100000:2.0:1785220581.678648:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510654848 00000100:00100000:2.0:1785220581.678653:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510654848 00000100:00100000:2.0:1785220581.678671:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510654720 00000100:00100000:2.0:1785220581.678674:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510654720 00000100:00100000:2.0:1785220581.678692:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510654592 00000100:00100000:2.0:1785220581.678695:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510654592 00000100:00100000:2.0:1785220581.678713:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284290 00000100:00100000:2.0:1785220581.678719:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1110:3654:x1871938510654336:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.678742:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29195 count 1 00000100:00100000:1.0:1785220581.678795:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284291 00000100:00100000:1.0:1785220581.678805:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1110:3654:x1871938510654848:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.678817:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29198 count 1 00010000:00010000:2.0:1785220581.678899:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678915:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003cf958cd/0xf62499ab92787f77 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8ae expref: 342 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678921:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x720b:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.678925:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.678929:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.678936:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.678942:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.678949:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.678952:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003cf958cd/0xf62499ab92787f77 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8ae expref: 342 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.678953:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.678963:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.678965:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003cf958cd/0xf62499ab92787f77 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8ae expref: 342 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.678975:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.678977:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510654976:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.678983:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4c58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.679000:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.679002:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.679003:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.679017:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.679028:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.679033:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284292 00000100:00100000:2.0:1785220581.679040:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1110:3653:x1871938510654720:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.679052:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29197 count 1 00010000:00010000:2.0:1785220581.679063:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.679067:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.679074:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a5821a04/0xf62499ab92788081 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b956 expref: 342 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679081:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x720d:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.679109:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.679114:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679133:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a5821a04/0xf62499ab92788081 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b956 expref: 342 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679142:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a5821a04/0xf62499ab92788081 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b956 expref: 342 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.679158:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510655104:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.679185:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.679203:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284293 00000100:00100000:2.0:1785220581.679210:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1110:3652:x1871938510654592:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.679215:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.679222:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29196 count 1 00010000:00010000:1.0:1785220581.679227:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd67aa20/0xf62499ab927880dc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9a3 expref: 342 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679232:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679241:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x720e:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.679243:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009b0d8cb1/0xf62499ab92787fee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8f4 expref: 342 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679243:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.679247:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679250:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x720c:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.679253:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.679256:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679265:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd67aa20/0xf62499ab927880dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9a3 expref: 342 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679269:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009b0d8cb1/0xf62499ab92787fee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8f4 expref: 342 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679274:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd67aa20/0xf62499ab927880dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9a3 expref: 342 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679278:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009b0d8cb1/0xf62499ab92787fee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8f4 expref: 342 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.679285:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510655232:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.679287:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510655360:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.679312:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df013c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.679331:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519935104 00000100:00100000:1.0:1785220581.679335:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519935104 00010000:00010000:1.0:1785220581.679354:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000070c16d85/0xf62499ab9278ab84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cebf expref: 305 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.679375:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519935232 00000100:00100000:1.0:1785220581.679378:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519935232 00010000:00010000:1.0:1785220581.679390:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e512a538/0xf62499ab9278a9ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cded expref: 306 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.679396:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.679409:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100219 00000100:00100000:1.0:1785220581.679415:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+306:2405:x1871938519935104:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.679424:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.679429:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000070c16d85/0xf62499ab9278ab84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cebf expref: 306 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.679431:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fd180/2, rc 0, transno 4295070549 00010000:00010000:1.0:1785220581.679439:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000070c16d85/0xf62499ab9278ab84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cebf expref: 306 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.679441:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510652928:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.679452:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070c16d85/0xf62499ab9278ab84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cebf expref: 306 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679462:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.679465:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.679469:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679477:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.679483:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72ce00/2, rc 0, transno 4295069964 00000100:00100000:2.0:1785220581.679493:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510652544:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.679495:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.679503:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679511:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.679513:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+305:2405:x1871938519935104:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (485us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.679520:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100219 00010000:00010000:2.0:1785220581.679521:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679529:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.679534:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100220 00000100:00100000:1.0:1785220581.679541:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+304:2407:x1871938519935232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.679542:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.679548:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.679553:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e512a538/0xf62499ab9278a9ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cded expref: 304 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679562:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e512a538/0xf62499ab9278a9ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cded expref: 304 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679573:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e512a538/0xf62499ab9278a9ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cded expref: 304 pid: 13867 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679581:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.679584:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.679587:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679594:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.679617:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.679634:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+303:2407:x1871938519935232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (566us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.679640:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100220 00000100:00100000:2.0:1785220581.679649:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1175:3653:x1871938510653824:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14185us (14305us total) trans 4295070550 rc 0/0 00000100:00100000:2.0:1785220581.679656:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284288 00000004:00080000:1.0:1785220581.679668:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99918000/2, rc 0, transno 4295070550 00010000:00010000:2.0:1785220581.679671:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.679676:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.679677:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510653824:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.679683:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.679687:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679695:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.679707:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.679708:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:2.0:1785220581.679714:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4c5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.679716:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510655616:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.679730:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.679731:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.679733:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.679741:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.679775:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510655744:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.679787:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be7b80/2, rc 0, transno 4295069958 00000100:00100000:2.0:1785220581.679795:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510652288:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.679796:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.679822:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510655616 00000004:00080000:2.0:1785220581.679824:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa1880/2, rc 0, transno 4295069961 00000100:00100000:1.0:1785220581.679825:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510655616 00000100:00100000:2.0:1785220581.679830:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510651136:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.679856:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa2680/2, rc 0, transno 4295069963 00000100:00100000:2.0:1785220581.679863:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510651392:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.679911:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510656128:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.679935:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.679952:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510655744 00000100:00100000:1.0:1785220581.679956:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510655744 00000004:00080000:2.0:1785220581.679967:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601f800/2, rc 0, transno 4295070547 00000100:00100000:1.0:1785220581.679970:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510656128 00000100:00100000:1.0:1785220581.679972:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510656128 00000100:00100000:2.0:1785220581.679974:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510650240:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.679987:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284294 00000100:00100000:1.0:1785220581.679992:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1175:3653:x1871938510655616:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.680001:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00002000:00080000:1.0:1785220581.680004:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29248 count 1 00000100:00100000:2.0:1785220581.680007:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510656256:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.680016:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680027:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.680029:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f15299c6/0xf62499ab9278adde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cff3 expref: 302 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680035:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7240:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.680038:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7240:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.680042:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.680053:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601e680/2, rc 0, transno 4295070546 00000100:00100000:2.0:1785220581.680060:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510650368:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.680062:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f15299c6/0xf62499ab9278adde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cff3 expref: 302 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680071:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f15299c6/0xf62499ab9278adde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cff3 expref: 302 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680081:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510655488:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.680121:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510656384:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.680220:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.680234:0:14261:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510656256 00000100:00100000:1.0:1785220581.680237:0:14261:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510656256 00000100:00100000:2.0:1785220581.680243:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.680260:0:14261:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510655488 00000100:00100000:2.0:1785220581.680261:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510656000:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.680262:0:14261:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510655488 00000100:00100000:2.0:1785220581.680279:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.680281:0:14261:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284295 00000100:00100000:1.0:1785220581.680287:0:14261:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1112:3653:x1871938510655744:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.680297:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510656000 00002000:00080000:1.0:1785220581.680298:0:14261:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29199 count 1 00000100:00100000:2.0:1785220581.680300:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510656000 00010000:00010000:1.0:1785220581.680310:0:14261:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680321:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284296 00010000:00010000:1.0:1785220581.680322:0:14261:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004f2c7bee/0xf62499ab92788176 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9db expref: 342 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680327:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1113:3652:x1871938510656128:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.680329:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x720f:0x0].0x0 since=28s v=(0 0) 00010000:00010000:1.0:1785220581.680332:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.680336:0:14261:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.680337:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29202 count 1 00010000:00010000:2.0:1785220581.680347:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680353:0:14261:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004f2c7bee/0xf62499ab92788176 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9db expref: 342 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680358:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000df7e2125/0xf62499ab927883ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb40 expref: 342 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680361:0:14261:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004f2c7bee/0xf62499ab92788176 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9db expref: 342 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680365:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7212:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.680368:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7212:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.680372:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.680378:0:14261:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510656640:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.680387:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000df7e2125/0xf62499ab927883ec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb40 expref: 342 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680396:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000df7e2125/0xf62499ab927883ec lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb40 expref: 342 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.680398:0:14261:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.680406:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510656768:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.680415:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284297 00000100:00100000:1.0:1785220581.680421:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1177:3652:x1871938510656256:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.680423:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00002000:00080000:1.0:1785220581.680432:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29249 count 1 00000100:00100000:2.0:1785220581.680436:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284298 00000100:00100000:2.0:1785220581.680441:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1177:3655:x1871938510655488:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.680443:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.680451:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29247 count 1 00010000:00010000:1.0:1785220581.680454:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a34eb71e/0xf62499ab9278ae8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d063 expref: 302 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680460:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7241:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.680461:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680463:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7241:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.680467:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680471:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ddabf54b/0xf62499ab9278ad28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf98 expref: 302 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680478:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x723f:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.680480:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.680483:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a34eb71e/0xf62499ab9278ae8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d063 expref: 302 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680484:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 4/0,1 mode: --/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680493:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a34eb71e/0xf62499ab9278ae8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d063 expref: 302 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680500:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ddabf54b/0xf62499ab9278ad28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf98 expref: 302 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.680502:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0000_UUID:14185:1871938510656896:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.680508:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ddabf54b/0xf62499ab9278ad28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf98 expref: 302 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.680522:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdb40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.680523:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90143480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510657024:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.680538:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.680538:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284299 00000100:00100000:1.0:1785220581.680544:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1113:3655:x1871938510656000:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.680549:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1785220581.680554:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510655872:0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.680555:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29201 count 1 00010000:00010000:1.0:1785220581.680565:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680575:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.680577:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002629dc37/0xf62499ab92788328 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bad0 expref: 342 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680583:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7211:0x0].0x0 since=28s v=(0 0) 00010000:00010000:1.0:1785220581.680586:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7211:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.680589:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680596:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510655872 00000100:00100000:2.0:1785220581.680599:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510655872 00010000:00010000:1.0:1785220581.680607:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002629dc37/0xf62499ab92788328 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bad0 expref: 342 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680616:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002629dc37/0xf62499ab92788328 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bad0 expref: 342 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680619:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284300 00000100:00100000:2.0:1785220581.680624:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1114:3655:x1871938510655872:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.680626:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510657152:192.168.201.51@tcp:104: 00002000:00080000:2.0:1785220581.680634:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29200 count 1 00000100:00100000:1.0:1785220581.680643:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd6c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.680645:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680655:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd3e7c6e/0xf62499ab9278825d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba52 expref: 342 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680662:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7210:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.680664:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7210:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.680668:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680686:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd3e7c6e/0xf62499ab9278825d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba52 expref: 342 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.680687:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.680700:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd3e7c6e/0xf62499ab9278825d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba52 expref: 342 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.680707:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510654464 00000100:00100000:2.0:1785220581.680709:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510657280:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.680710:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510654464 00000100:00100000:2.0:1785220581.680727:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.680731:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284301 00000100:00100000:1.0:1785220581.680737:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1178:3654:x1871938510654464:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.680741:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510656512:0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.680748:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29246 count 1 00010000:00010000:1.0:1785220581.680770:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680773:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.680782:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006a286d62/0xf62499ab9278ac56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf4b expref: 302 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.680789:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x723e:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.680792:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723e:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220581.680793:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510656512 00010000:00010000:1.0:1785220581.680795:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680796:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510656512 00010000:00010000:1.0:1785220581.680813:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006a286d62/0xf62499ab9278ac56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf4b expref: 302 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680818:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284302 00000100:00100000:2.0:1785220581.680823:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1179:3655:x1871938510656512:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.680823:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006a286d62/0xf62499ab9278ac56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf4b expref: 302 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.680833:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29250 count 1 00010000:00010000:2.0:1785220581.680844:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680854:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007fcec859/0xf62499ab9278af7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d0da expref: 302 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680860:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7242:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.680863:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7242:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.680867:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680885:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007fcec859/0xf62499ab9278af7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d0da expref: 302 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.680895:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007fcec859/0xf62499ab9278af7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d0da expref: 302 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.680904:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510657536:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.680905:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510657408:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.680922:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df180 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.681638:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd480 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.681730:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.681742:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.681751:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.681770:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.682512:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.682520:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.682534:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.682538:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.682548:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.682568:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.682577:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.682579:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.682580:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.682606:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1785220581.682612:0:14259:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4c66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.682902:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1180:3654:x1871938510653568:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17436us (17643us total) trans 4295070551 rc 0/0 00000100:00100000:0.0:1785220581.682912:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284287 00000100:00100000:2.0:1785220581.683009:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.683433:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.683479:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0000_UUID:14266:1871938510654208:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.683504:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.683548:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991bb80/2, rc 0, transno 4295070551 00000100:00100000:1.0:1785220581.683561:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510653568:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.683650:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.683668:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510657664:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.683706:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.683818:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510654976:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.683835:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.683890:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.683910:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519935360 00000100:00100000:3.0:1785220581.683915:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519935360 00010000:00010000:3.0:1785220581.683947:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000056dd9e53/0xf62499ab9278a87b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd0d expref: 303 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.683972:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519935488 00000100:00100000:3.0:1785220581.683975:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519935488 00010000:00010000:3.0:1785220581.683986:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cae27715/0xf62499ab9278aace lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce56 expref: 304 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.684007:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100221 00000100:00100000:3.0:1785220581.684017:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+304:2404:x1871938519935360:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.684030:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.684036:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000056dd9e53/0xf62499ab9278a87b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd0d expref: 304 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.684047:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000056dd9e53/0xf62499ab9278a87b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd0d expref: 304 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.684067:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510657664 00010000:00010000:3.0:1785220581.684068:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056dd9e53/0xf62499ab9278a87b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd0d expref: 304 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.684072:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510657664 00010000:00010000:3.0:1785220581.684083:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7239:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.684317:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284303 00000100:00100000:0.0:1785220581.684326:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1180:3654:x1871938510657664:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.684346:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29251 count 1 00010000:00010000:0.0:1785220581.684366:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.684379:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001f019b45/0xf62499ab9278b07e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d158 expref: 303 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.684385:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7243:0x0].0x0 since=26s v=(0 0) 00010000:00010000:0.0:1785220581.684389:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7243:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.684393:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.684418:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001f019b45/0xf62499ab9278b07e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d158 expref: 303 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.684429:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001f019b45/0xf62499ab9278b07e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d158 expref: 303 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.684442:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510657792:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.684496:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.685046:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7239:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.685052:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685063:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685109:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.685125:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685139:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685149:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685159:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.685163:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+303:2404:x1871938519935360:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1145us (2153us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.685172:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100221 00010000:00010000:0.0:1785220581.685178:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.685200:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100222 00000100:00100000:3.0:1785220581.685208:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+302:2406:x1871938519935488:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.685222:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.685228:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cae27715/0xf62499ab9278aace lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce56 expref: 302 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685237:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cae27715/0xf62499ab9278aace lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce56 expref: 302 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685252:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cae27715/0xf62499ab9278aace lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce56 expref: 302 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685264:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.685266:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.685271:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685278:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685292:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.685309:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+301:2406:x1871938519935488:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (1877us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.685315:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100222 00000100:00100000:0.0:1785220581.685333:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1181:3653:x1871938510652800:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24326us (24665us total) trans 4295070552 rc 0/0 00000100:00100000:0.0:1785220581.685340:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284285 00010000:00010000:0.0:1785220581.685360:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.685366:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685374:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.685378:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685387:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685401:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685409:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4c51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685426:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.685428:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.685430:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.685476:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.685491:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.685500:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.685507:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.685528:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.685550:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519935616 00000100:00100000:3.0:1785220581.685555:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519935616 00010000:00010000:3.0:1785220581.685578:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003cf958cd/0xf62499ab92787f77 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8ae expref: 343 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.685606:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100223 00000100:00100000:1.0:1785220581.685607:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1181:3653:x1871938510653696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19972us (20206us total) trans 4295070553 rc 0/0 00000100:00100000:3.0:1785220581.685612:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+343:2406:x1871938519935616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.685615:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284289 00010000:00010000:3.0:1785220581.685620:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.685624:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003cf958cd/0xf62499ab92787f77 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8ae expref: 343 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685634:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003cf958cd/0xf62499ab92787f77 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8ae expref: 343 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685648:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003cf958cd/0xf62499ab92787f77 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8ae expref: 343 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685657:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720b:0x0].0x0 since=0s v=(1 0) 00000004:00080000:1.0:1785220581.685657:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99918e00/2, rc 0, transno 4295070553 00010000:00010000:3.0:1785220581.685660:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.685664:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.685673:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.685676:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510653696:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.685688:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1785220581.685688:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93609c00/2, rc 0, transno 4295070552 00000100:00100000:1.0:1785220581.685692:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510652800:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.685707:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+342:2406:x1871938519935616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1817us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.685714:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100223 00010000:00010000:2.0:1785220581.685757:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.685775:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.685785:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.685793:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.685815:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.685918:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1115:3654:x1871938510654336:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7198us (8256us total) trans 4295069965 rc 0/0 00000100:00100000:2.0:1785220581.685925:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284290 00010000:00010000:0.0:1785220581.685970:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.685975:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685983:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.685986:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.685995:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.686002:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510658048:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.686006:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686012:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a4c6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686027:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.686031:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686037:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.686040:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686047:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.686055:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.686057:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686064:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4c7b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686077:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.686078:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.686079:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:0.0:1785220581.686080:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.686107:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510657920:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.686109:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510658048 00000100:00100000:0.0:1785220581.686112:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510658048 00000100:00100000:2.0:1785220581.686137:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.686144:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510657920 00000100:00100000:0.0:1785220581.686147:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510657920 00000100:00100000:0.0:1785220581.686166:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284304 00000100:00100000:0.0:1785220581.686172:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d935a2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1182:3653:x1871938510658048:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.686190:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29253 count 1 00010000:00010000:0.0:1785220581.686208:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686227:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007f5d0c11/0xf62499ab9278b229 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d231 expref: 300 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686234:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7245:0x0].0x0 since=26s v=(0 0) 00010000:00010000:0.0:1785220581.686237:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7245:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.686241:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686262:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007f5d0c11/0xf62499ab9278b229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d231 expref: 300 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686273:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007f5d0c11/0xf62499ab9278b229 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d231 expref: 300 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.686275:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.686285:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510658176:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.686334:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26a80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.686350:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284305 00000100:00100000:0.0:1785220581.686359:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1182:3653:x1871938510657920:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.686371:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29252 count 1 00010000:00010000:0.0:1785220581.686382:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686394:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd0ec9de/0xf62499ab9278b150 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1c1 expref: 300 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686405:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7244:0x0].0x0 since=26s v=(0 0) 00010000:00010000:0.0:1785220581.686408:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7244:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.686412:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686431:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fd0ec9de/0xf62499ab9278b150 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1c1 expref: 300 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.686442:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fd0ec9de/0xf62499ab9278b150 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1c1 expref: 300 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.686451:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510658304:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.686474:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26cc0 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.686629:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083d180/2, rc 0, transno 4295069965 00000100:00100000:1.0:1785220581.686643:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510654336:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.687359:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510658432:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.687396:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.689574:0:14186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510658432 00000100:00100000:3.0:1785220581.689587:0:14186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510658432 00000100:00100000:3.0:1785220581.689683:0:14186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284306 00000100:00100000:3.0:1785220581.689693:0:14186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1115:3654:x1871938510658432:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.689722:0:14186:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29203 count 1 00010000:00010000:3.0:1785220581.689746:0:14186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.689776:0:14186:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c08f5f57/0xf62499ab92788494 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bba2 expref: 341 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.689776:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510655232:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.689785:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7213:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.689789:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7213:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.689794:0:14186:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.689798:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.689822:0:14186:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c08f5f57/0xf62499ab92788494 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bba2 expref: 341 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.689834:0:14186:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c08f5f57/0xf62499ab92788494 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bba2 expref: 341 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.689849:0:14186:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510658560:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.689854:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.689872:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.689910:0:14186:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.689923:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510655104:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.689937:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.689957:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519935872 00000100:00100000:0.0:1785220581.689960:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519935872 00010000:00010000:0.0:1785220581.689982:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd67aa20/0xf62499ab927880dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9a3 expref: 342 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.690010:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519935744 00000100:00100000:0.0:1785220581.690013:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519935744 00010000:00010000:0.0:1785220581.690026:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a5821a04/0xf62499ab92788081 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b956 expref: 343 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.690056:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100224 00000100:00100000:0.0:1785220581.690064:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+343:2406:x1871938519935872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.690075:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.690080:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd67aa20/0xf62499ab927880dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9a3 expref: 343 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.690115:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd67aa20/0xf62499ab927880dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9a3 expref: 343 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.690146:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd67aa20/0xf62499ab927880dc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9a3 expref: 343 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.690160:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.690164:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.690168:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.690177:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.690195:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.690235:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100225 00000100:00100000:3.0:1785220581.690244:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90700050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+342:2406:x1871938519935744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.690253:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.690258:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a5821a04/0xf62499ab92788081 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b956 expref: 342 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.690269:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a5821a04/0xf62499ab92788081 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b956 expref: 342 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.690282:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5821a04/0xf62499ab92788081 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b956 expref: 342 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.690292:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.690295:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.690299:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.690306:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.690323:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.690343:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90700050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+341:2406:x1871938519935744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (469us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.690351:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100225 00000100:00100000:1.0:1785220581.690364:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.690381:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936000 00000100:00100000:1.0:1785220581.690385:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936000 00010000:00010000:1.0:1785220581.690402:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009b0d8cb1/0xf62499ab92787fee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8f4 expref: 341 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.690425:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100226 00000100:00100000:1.0:1785220581.690432:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+341:2406:x1871938519936000:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.690440:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.690444:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009b0d8cb1/0xf62499ab92787fee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8f4 expref: 341 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.690454:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009b0d8cb1/0xf62499ab92787fee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8f4 expref: 341 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.690466:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.690469:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.690472:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.690479:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.690494:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.690513:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+341:2406:x1871938519936000:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (148us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.690520:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100226 00000100:00100000:1.0:1785220581.690588:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510656384:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.690603:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.690683:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510656640:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.690695:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.690708:0:14187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009b0d8cb1/0xf62499ab92787fee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b8f4 expref: 340 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.690719:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510655360:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.690730:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.690739:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.690746:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.690754:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.690776:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.690798:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.690952:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1116:3652:x1871938510654592:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11742us (13123us total) trans 4295069966 rc 0/0 00000100:00100000:2.0:1785220581.690960:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284293 00000004:00080000:2.0:1785220581.690997:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083fb80/2, rc 0, transno 4295069966 00010000:00010000:1.0:1785220581.691005:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1785220581.691010:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510654592:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.691011:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.691019:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.691023:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.691031:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.691042:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.691049:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4c89 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.691066:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.691068:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.691069:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.691080:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.691112:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510658688:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.691202:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.691694:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+339:2406:x1871938519935872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1628us (1836us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.691706:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100224 00010000:00010000:0.0:1785220581.691778:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.691787:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.691796:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.691805:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.691829:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.691949:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1116:3653:x1871938510654720:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12909us (14137us total) trans 4295069967 rc 0/0 00000100:00100000:0.0:1785220581.691957:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284292 00000100:00100000:0.0:1785220581.691972:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510658688 00000100:00100000:0.0:1785220581.691977:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510658688 00000100:00100000:0.0:1785220581.692007:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284307 00000100:00100000:0.0:1785220581.692015:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1116:3654:x1871938510658688:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.692040:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29204 count 1 00010000:00010000:0.0:1785220581.692061:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692076:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bb32611c/0xf62499ab92788551 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc27 expref: 338 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692109:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7214:0x0].0x0 since=28s v=(0 0) 00010000:00010000:0.0:1785220581.692114:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7214:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.692119:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692149:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bb32611c/0xf62499ab92788551 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc27 expref: 338 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.692162:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083c700/2, rc 0, transno 4295069967 00010000:00010000:0.0:1785220581.692163:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bb32611c/0xf62499ab92788551 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc27 expref: 338 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.692172:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510654720:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.692178:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510658816:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.692216:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.692220:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.692224:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510658944:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.692237:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692245:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.692250:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.692254:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692261:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.692274:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510658944 00000100:00100000:2.0:1785220581.692278:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510658944 00010000:00010000:0.0:1785220581.692279:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.692299:0:13869:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284308 00000100:00100000:2.0:1785220581.692305:0:13869:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1117:3653:x1871938510658944:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.692316:0:13869:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29205 count 1 00010000:00010000:2.0:1785220581.692329:0:13869:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.692341:0:13869:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000095084431/0xf62499ab9278861c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc89 expref: 338 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.692353:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7215:0x0].0x0 since=28s v=(0 0) 00000100:00100000:0.0:1785220581.692354:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1118:3654:x1871938510654848:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13552us (14565us total) trans 4295069968 rc 0/0 00010000:00010000:2.0:1785220581.692356:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7215:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.692360:0:13869:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.692361:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284291 00010000:00010000:2.0:1785220581.692379:0:13869:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000095084431/0xf62499ab9278861c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc89 expref: 338 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692380:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.692385:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.692389:0:13869:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000095084431/0xf62499ab9278861c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc89 expref: 338 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692393:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.692397:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.692399:0:13869:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510659072:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.692406:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.692418:0:13869:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.692422:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692429:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a4c82 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692445:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.692448:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692460:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.692463:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692470:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692479:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692486:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a4c74 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.692499:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.692501:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.692503:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.692904:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.692930:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083ca80/2, rc 0, transno 4295069968 00000100:00100000:1.0:1785220581.692949:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510654848:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.693171:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220581.693182:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510659200:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.693216:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.693239:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510659200 00000100:00100000:2.0:1785220581.693243:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510659200 00000100:00100000:2.0:1785220581.693270:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284309 00000100:00100000:2.0:1785220581.693276:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1118:3652:x1871938510659200:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.693291:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29206 count 1 00010000:00010000:2.0:1785220581.693308:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.693329:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e1f7564c/0xf62499ab9278871f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd2a expref: 338 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.693338:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7216:0x0].0x0 since=28s v=(0 0) 00010000:00010000:2.0:1785220581.693341:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7216:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.693346:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.693368:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e1f7564c/0xf62499ab9278871f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd2a expref: 338 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.693379:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e1f7564c/0xf62499ab9278871f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd2a expref: 338 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.693391:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510659328:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.693434:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.693566:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.693590:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936128 00000100:00100000:1.0:1785220581.693594:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936128 00010000:00010000:1.0:1785220581.693616:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f15299c6/0xf62499ab9278adde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cff3 expref: 301 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.693645:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100227 00000100:00100000:1.0:1785220581.693652:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+301:2404:x1871938519936128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.693663:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.693668:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f15299c6/0xf62499ab9278adde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cff3 expref: 301 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.693679:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f15299c6/0xf62499ab9278adde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cff3 expref: 301 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.693699:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f15299c6/0xf62499ab9278adde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cff3 expref: 301 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.693710:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7240:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.693712:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7240:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.693718:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.693727:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.693766:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.693788:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.693789:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+300:2404:x1871938519936128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 137us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.693796:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100227 00010000:00010000:2.0:1785220581.693797:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.693806:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.693819:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.693859:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.693871:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.693976:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510657408:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.693992:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.694007:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1183:3653:x1871938510655616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14014us (14264us total) trans 4295070554 rc 0/0 00000100:00100000:2.0:1785220581.694014:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284294 00000100:00100000:1.0:1785220581.694031:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.694045:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510656768:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.694049:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.694059:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.694064:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.694102:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510657536:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.694115:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.694120:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.694191:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0000_UUID:14185:1871938510656896:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.694196:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510657152:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.694203:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.694206:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.694225:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90143480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510657024:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.694235:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.694260:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510657280:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.694272:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.694462:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510657792:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.694476:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.695382:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936256 00000100:00100000:2.0:1785220581.695389:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936256 00000100:00100000:0.0:1785220581.695396:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936512 00000100:00100000:0.0:1785220581.695400:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936512 00010000:00010000:2.0:1785220581.695422:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004f2c7bee/0xf62499ab92788176 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9db expref: 339 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.695422:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007fcec859/0xf62499ab9278af7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d0da expref: 300 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.695452:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936768 00000100:00100000:0.0:1785220581.695455:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936384 00000100:00100000:2.0:1785220581.695456:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936768 00000100:00100000:0.0:1785220581.695457:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936384 00010000:00010000:2.0:1785220581.695469:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001f019b45/0xf62499ab9278b07e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d158 expref: 301 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.695469:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000df7e2125/0xf62499ab927883ec lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb40 expref: 340 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.695487:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936640 00000100:00100000:0.0:1785220581.695490:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936640 00010000:00010000:0.0:1785220581.695502:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006a286d62/0xf62499ab9278ac56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf4b expref: 302 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.695736:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.695744:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.695769:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.695775:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.695785:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.695804:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.695812:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4c90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.695833:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.695836:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.695837:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.697068:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100228 00000100:00100000:0.0:1785220581.697239:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+340:2404:x1871938519936256:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.697258:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100229 00010000:00010000:0.0:1785220581.697261:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.697267:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+302:2407:x1871938519936512:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.697270:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004f2c7bee/0xf62499ab92788176 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9db expref: 340 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697276:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.697281:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007fcec859/0xf62499ab9278af7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d0da expref: 302 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697286:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004f2c7bee/0xf62499ab92788176 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9db expref: 340 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697290:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007fcec859/0xf62499ab9278af7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d0da expref: 302 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697310:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007fcec859/0xf62499ab9278af7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d0da expref: 302 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697310:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f2c7bee/0xf62499ab92788176 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b9db expref: 340 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697327:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7242:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.697327:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.697332:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7242:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.697332:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x720f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.697339:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697340:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697351:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697351:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697378:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.697378:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.697440:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100230 00000100:00100000:1.0:1785220581.697450:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+339:2407:x1871938519936384:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.697459:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.697465:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000df7e2125/0xf62499ab927883ec lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb40 expref: 339 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.697475:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000df7e2125/0xf62499ab927883ec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb40 expref: 339 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.697487:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df7e2125/0xf62499ab927883ec lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bb40 expref: 339 pid: 14187 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.697497:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7212:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.697500:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7212:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.697503:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.697510:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.697527:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.697563:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100231 00000100:00100000:3.0:1785220581.697573:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+301:2407:x1871938519936768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.697579:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb20700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+338:2404:x1871938519936256:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 502us (3720us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.697582:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220581.697587:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100228 00010000:00010000:3.0:1785220581.697588:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001f019b45/0xf62499ab9278b07e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d158 expref: 301 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.697597:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001f019b45/0xf62499ab9278b07e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d158 expref: 301 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.697606:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100232 00000100:00100000:0.0:1785220581.697613:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+301:2407:x1871938519936640:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.697614:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f019b45/0xf62499ab9278b07e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d158 expref: 301 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697620:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.697624:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006a286d62/0xf62499ab9278ac56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf4b expref: 300 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.697625:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7243:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.697628:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7243:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.697634:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697635:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006a286d62/0xf62499ab9278ac56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf4b expref: 300 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.697643:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697647:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a286d62/0xf62499ab9278ac56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf4b expref: 300 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697656:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.697659:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.697661:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.697663:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697675:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697691:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.697697:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+299:2407:x1871938519936512:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 431us (3665us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.697703:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100229 00000100:00100000:0.0:1785220581.697705:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+299:2407:x1871938519936640:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (3620us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.697710:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100232 00010000:00010000:0.0:1785220581.697739:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697748:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697749:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697757:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697772:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697777:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.697782:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697786:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.697802:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220581.697807:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.698020:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1119:3652:x1871938510656128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17691us (18082us total) trans 4295069969 rc 0/0 00000100:00100000:0.0:1785220581.698021:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1183:3654:x1871938510657664:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13698us (14315us total) trans 4295070555 rc 0/0 00000100:00100000:0.0:1785220581.698030:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284303 00010000:00010000:0.0:1785220581.698053:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:3.0:1785220581.698056:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+297:2407:x1871938519936768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 485us (4005us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.698059:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.698062:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100231 00010000:00010000:0.0:1785220581.698068:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.698072:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.698081:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.698244:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2407:x1871938519936384:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 796us (4178us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.698251:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100230 00000004:00080000:3.0:1785220581.698295:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb20000/2, rc 0, transno 4295070554 00010000:00010000:1.0:1785220581.698297:0:14261:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.698305:0:14261:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.698307:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510655616:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.698314:0:14261:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.698321:0:14261:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.698333:0:14261:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.698375:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284296 00010000:00010000:2.0:1785220581.698395:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.698400:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698409:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ada00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.698412:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698420:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.698425:0:14261:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1119:3653:x1871938510655744:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18139us (18628us total) trans 4295069970 rc 0/0 00000100:00100000:1.0:1785220581.698432:0:14261:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284295 00010000:00010000:2.0:1785220581.698434:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698441:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a4ccf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.698450:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698456:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.698458:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698459:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698467:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.698467:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698470:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.698481:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698484:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698494:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.698495:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.698500:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4c97 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x720f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698513:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.698516:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698522:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.698525:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698532:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698542:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698549:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4cc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.698561:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1183:3654:x1871938510654464:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17825us (17874us total) trans 4295070556 rc 0/0 00010000:00010000:2.0:1785220581.698567:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.698567:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284301 00010000:00010000:2.0:1785220581.698569:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.698570:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.698580:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.698581:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.698585:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.698589:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698597:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698604:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.698605:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb23480/2, rc 0, transno 4295070555 00000100:00100000:1.0:1785220581.698609:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510657664:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.698616:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1785220581.698617:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fe680/2, rc 0, transno 4295070556 00000100:00100000:1.0:1785220581.698620:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510654464:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.698688:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1183:3655:x1871938510656512:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17865us (17914us total) trans 4295070557 rc 0/0 00000100:00100000:2.0:1785220581.698695:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284302 00010000:00010000:2.0:1785220581.698711:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.698715:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698722:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.698725:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698734:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.698744:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.698750:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.698751:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4cc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.698797:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.698807:0:14186:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510658560:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.698826:0:14186:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.698852:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fea00/2, rc 0, transno 4295069970 00000100:00100000:3.0:1785220581.698863:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510655744:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.698904:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.698906:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.698907:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.698925:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.698932:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a4c9e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7212:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.698945:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.698946:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.698948:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.698960:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937280 00000100:00100000:0.0:1785220581.698966:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937280 00000100:00100000:3.0:1785220581.698984:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510659456:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.698988:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002629dc37/0xf62499ab92788328 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bad0 expref: 337 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.699008:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519936896 00000100:00100000:0.0:1785220581.699011:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519936896 00000100:00100000:3.0:1785220581.699014:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.699024:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ddabf54b/0xf62499ab9278ad28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf98 expref: 297 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.699060:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100233 00000100:00100000:0.0:1785220581.699067:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2405:x1871938519937280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.699081:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220581.699082:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:0.0:1785220581.699220:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002629dc37/0xf62499ab92788328 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bad0 expref: 337 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.699230:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002629dc37/0xf62499ab92788328 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bad0 expref: 337 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.699245:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002629dc37/0xf62499ab92788328 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bad0 expref: 337 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.699256:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7211:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.699259:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7211:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.699264:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.699272:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.699293:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.699297:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510659584:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.699315:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+336:2405:x1871938519937280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 248us (565us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.699318:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.699322:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100233 00000100:00100000:0.0:1785220581.699336:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100234 00000100:00100000:3.0:1785220581.699340:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510659456 00000100:00100000:0.0:1785220581.699342:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+297:2405:x1871938519936896:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.699343:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510659456 00010000:00010000:0.0:1785220581.699350:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.699355:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ddabf54b/0xf62499ab9278ad28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf98 expref: 297 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.699364:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510659584 00010000:00010000:0.0:1785220581.699365:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ddabf54b/0xf62499ab9278ad28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf98 expref: 297 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.699367:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510659584 00010000:00010000:0.0:1785220581.699379:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddabf54b/0xf62499ab9278ad28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf98 expref: 297 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.699381:0:14170:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284310 00000100:00100000:3.0:1785220581.699387:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+1184:3654:x1871938510659456:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.699389:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.699392:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x723f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.699395:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.699402:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.699407:0:14170:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29254 count 1 00010000:00010000:0.0:1785220581.699417:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.699428:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.699433:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+296:2405:x1871938519936896:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (636us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.699439:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100234 00010000:00010000:3.0:1785220581.699441:0:14170:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000063d132a0/0xf62499ab9278b3aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2ee expref: 296 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.699448:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7246:0x0].0x0 since=26s v=(0 0) 00010000:00010000:3.0:1785220581.699450:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7246:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.699454:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.699471:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510658304:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.699474:0:14170:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000063d132a0/0xf62499ab9278b3aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2ee expref: 295 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.699484:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.699485:0:14170:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000063d132a0/0xf62499ab9278b3aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2ee expref: 295 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.699497:0:14170:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510659712:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.699506:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510658176:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.699516:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.699532:0:14170:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.699556:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93664700/2, rc 0, transno 4295070557 00000100:00100000:0.0:1785220581.699560:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510656512:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.699665:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.699720:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510659968:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.699748:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.700012:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700022:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700031:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700039:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700053:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.700419:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510659840:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.700449:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.700487:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510659968 00000100:00100000:2.0:1785220581.700494:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510659968 00000100:00100000:2.0:1785220581.700537:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284311 00000100:00100000:2.0:1785220581.700545:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1185:3654:x1871938510659584:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.700564:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29255 count 1 00010000:00010000:2.0:1785220581.700581:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.700598:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bafaa249/0xf62499ab9278b4f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d381 expref: 295 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.700606:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7247:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.700609:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7247:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.700613:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.700636:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bafaa249/0xf62499ab9278b4f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d381 expref: 295 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.700646:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bafaa249/0xf62499ab9278b4f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d381 expref: 295 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.700655:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1119:3655:x1871938510656000:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20111us (20375us total) trans 4295069971 rc 0/0 00000100:00100000:2.0:1785220581.700657:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90143480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510660096:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.700687:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284312 00000100:00100000:0.0:1785220581.700696:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1185:3653:x1871938510659968:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.700709:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29257 count 1 00010000:00010000:0.0:1785220581.700720:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.700721:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.700731:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d99f2dff/0xf62499ab9278b60b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d429 expref: 295 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.700736:0:14267:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510659840 00010000:00010000:0.0:1785220581.700738:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7249:0x0].0x0 since=25s v=(0 0) 00000100:00100000:2.0:1785220581.700739:0:14267:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510659840 00010000:00010000:0.0:1785220581.700741:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7249:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.700745:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.700774:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284313 00010000:00010000:0.0:1785220581.700774:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d99f2dff/0xf62499ab9278b60b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d429 expref: 295 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.700780:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1186:3653:x1871938510659840:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.700790:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29256 count 1 00010000:00010000:0.0:1785220581.700790:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d99f2dff/0xf62499ab9278b60b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d429 expref: 295 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.700800:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510660224:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.700801:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.700811:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c6a41fed/0xf62499ab9278b55c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3b9 expref: 295 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.700818:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7248:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.700820:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7248:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.700824:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.700837:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284299 00010000:00010000:2.0:1785220581.700847:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c6a41fed/0xf62499ab9278b55c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3b9 expref: 295 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.700856:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c6a41fed/0xf62499ab9278b55c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3b9 expref: 295 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.700866:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510660352:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.700866:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.700871:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700879:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.700883:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700892:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700911:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700919:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4cb3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7211:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.700935:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.700937:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.700939:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.700969:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.700990:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937536 00000100:00100000:1.0:1785220581.700993:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937536 00010000:00010000:1.0:1785220581.701010:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fd0ec9de/0xf62499ab9278b150 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1c1 expref: 296 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.701040:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100235 00000100:00100000:1.0:1785220581.701046:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+296:2405:x1871938519937536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.701054:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.701058:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fd0ec9de/0xf62499ab9278b150 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1c1 expref: 296 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701068:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fd0ec9de/0xf62499ab9278b150 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1c1 expref: 296 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701081:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd0ec9de/0xf62499ab9278b150 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1c1 expref: 296 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701117:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7244:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.701121:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7244:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.701125:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701207:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701226:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.701245:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+295:2405:x1871938519937536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 199us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.701251:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100235 00000100:00100000:1.0:1785220581.701300:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.701336:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.701359:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.701382:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.701396:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937024 00000100:00100000:1.0:1785220581.701399:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937024 00010000:00010000:1.0:1785220581.701413:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a34eb71e/0xf62499ab9278ae8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d063 expref: 295 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.701429:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100236 00000100:00100000:1.0:1785220581.701435:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+295:2406:x1871938519937024:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.701441:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.701444:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a34eb71e/0xf62499ab9278ae8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d063 expref: 295 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701453:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a34eb71e/0xf62499ab9278ae8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d063 expref: 295 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701465:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a34eb71e/0xf62499ab9278ae8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d063 expref: 295 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701473:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7241:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.701476:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7241:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.701481:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701488:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701502:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.701517:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+294:2406:x1871938519937024:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (216us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.701523:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100236 00000100:00100000:1.0:1785220581.701532:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937408 00000100:00100000:1.0:1785220581.701535:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937408 00000100:00100000:3.0:1785220581.701540:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937152 00000100:00100000:3.0:1785220581.701544:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937152 00010000:00010000:1.0:1785220581.701546:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007f5d0c11/0xf62499ab9278b229 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d231 expref: 294 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701560:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd3e7c6e/0xf62499ab9278825d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba52 expref: 336 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.701563:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937664 00000100:00100000:1.0:1785220581.701565:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937664 00010000:00010000:1.0:1785220581.701576:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c08f5f57/0xf62499ab92788494 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bba2 expref: 337 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.701580:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100237 00000100:00100000:3.0:1785220581.701587:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+294:2406:x1871938519937408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.701592:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100238 00010000:00010000:3.0:1785220581.701596:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220581.701597:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2406:x1871938519937152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.701600:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007f5d0c11/0xf62499ab9278b229 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d231 expref: 294 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701603:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.701607:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd3e7c6e/0xf62499ab9278825d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba52 expref: 337 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701610:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007f5d0c11/0xf62499ab9278b229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d231 expref: 294 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701616:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd3e7c6e/0xf62499ab9278825d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba52 expref: 337 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701623:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f5d0c11/0xf62499ab9278b229 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d231 expref: 294 pid: 14208 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701628:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd3e7c6e/0xf62499ab9278825d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba52 expref: 337 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701639:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7245:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.701639:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7210:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.701643:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7245:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.701643:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7210:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.701648:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701648:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701657:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701657:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701673:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.701673:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.701695:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+293:2406:x1871938519937408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (353us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.701701:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100237 00000100:00100000:3.0:1785220581.701718:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100239 00000100:00100000:3.0:1785220581.701724:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+336:2406:x1871938519937664:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.701731:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.701736:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c08f5f57/0xf62499ab92788494 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bba2 expref: 336 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701746:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c08f5f57/0xf62499ab92788494 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bba2 expref: 336 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701768:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c08f5f57/0xf62499ab92788494 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bba2 expref: 336 pid: 14185 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701784:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7213:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.701788:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7213:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.701792:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701799:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701817:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.701836:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+335:2406:x1871938519937664:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.701842:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100239 00010000:00010000:3.0:1785220581.701861:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701870:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701879:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.701887:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.701895:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+334:2406:x1871938519937152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 297us (534us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.701901:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.701901:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100238 00010000:00010000:1.0:1785220581.701938:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.701944:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701951:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.701954:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701962:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701977:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701984:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4ce4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7213:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.701999:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.702000:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.702002:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.702015:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1119:3654:x1871938510658432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12324us (14618us total) trans 4295069972 rc 0/0 00000100:00100000:3.0:1785220581.702022:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284306 00000004:00080000:3.0:1785220581.702064:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99919500/2, rc 0, transno 4295069972 00000100:00100000:3.0:1785220581.702070:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510658432:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.702148:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.702158:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.702166:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.702173:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.702186:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.702258:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.702265:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1187:3655:x1871938510655488:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21824us (22021us total) trans 4295070558 rc 0/0 00000100:00100000:3.0:1785220581.702272:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284298 00000004:00080000:3.0:1785220581.702302:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97841f80/2, rc 0, transno 4295069969 00000100:00100000:3.0:1785220581.702307:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510656128:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.702377:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.702387:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510660736:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.702561:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.702584:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.702796:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26a80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.702819:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937792 00000100:00100000:0.0:1785220581.702823:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937792 00010000:00010000:0.0:1785220581.702857:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000095084431/0xf62499ab9278861c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc89 expref: 334 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.702879:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938048 00000100:00100000:0.0:1785220581.702882:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938048 00010000:00010000:0.0:1785220581.702895:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e1f7564c/0xf62499ab9278871f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd2a expref: 335 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.702910:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519937920 00000100:00100000:0.0:1785220581.702913:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519937920 00010000:00010000:0.0:1785220581.702924:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bb32611c/0xf62499ab92788551 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc27 expref: 336 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.702944:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100240 00000100:00100000:0.0:1785220581.702952:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+336:2406:x1871938519937792:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.702961:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.702965:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000095084431/0xf62499ab9278861c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc89 expref: 336 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.702977:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000095084431/0xf62499ab9278861c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc89 expref: 336 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.702993:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7215:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.702997:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7215:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.703002:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703011:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703025:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.703044:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+336:2406:x1871938519937792:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (787us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.703050:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100240 00000100:00100000:0.0:1785220581.703064:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100241 00000100:00100000:0.0:1785220581.703070:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+335:2406:x1871938519938048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.703078:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.703106:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e1f7564c/0xf62499ab9278871f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd2a expref: 335 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703117:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e1f7564c/0xf62499ab9278871f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd2a expref: 335 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703770:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7216:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.703775:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7216:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.703780:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703789:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703804:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.703831:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+335:2406:x1871938519938048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 760us (1269us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.703838:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100241 00000100:00100000:0.0:1785220581.703857:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100242 00000100:00100000:0.0:1785220581.703865:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+334:2406:x1871938519937920:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.703874:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.703879:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bb32611c/0xf62499ab92788551 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc27 expref: 334 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703888:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bb32611c/0xf62499ab92788551 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc27 expref: 334 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703901:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7214:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.703904:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7214:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.703908:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703916:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.703928:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.703945:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+334:2406:x1871938519937920:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (1360us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.703951:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100242 00000100:00100000:2.0:1785220581.703977:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.703997:0:14245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb32611c/0xf62499ab92788551 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc27 expref: 333 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.703999:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704008:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.704011:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510658816:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.704017:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704023:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704033:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704040:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220581.704041:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704047:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704053:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704060:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704075:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.704213:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1119:3655:x1871938510655872:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23587us (23635us total) trans 4295069973 rc 0/0 00000100:00100000:2.0:1785220581.704220:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284300 00010000:00010000:2.0:1785220581.704244:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704253:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704261:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704268:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704282:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.704335:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1119:3654:x1871938510658688:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12320us (13132us total) trans 4295069974 rc 0/0 00000100:00100000:0.0:1785220581.704342:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284307 00000100:00100000:2.0:1785220581.704360:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1187:3652:x1871938510656256:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23940us (24332us total) trans 4295070559 rc 0/0 00010000:00010000:0.0:1785220581.704361:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00000100:00100000:2.0:1785220581.704367:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284297 00010000:00010000:0.0:1785220581.704369:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704377:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.704382:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704386:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.704390:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704391:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704404:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 1/0,0 mode: --/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704411:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a4cac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704432:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.704436:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704446:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.704449:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704455:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704463:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704470:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a4ceb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7214:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704482:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.704485:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704492:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.704495:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704502:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704511:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704518:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4ca5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704531:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.704532:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.704534:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.704545:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704553:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704560:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704567:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704580:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.704588:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.704593:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704602:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704618:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704625:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a4cba lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7210:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704641:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.704644:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.704647:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1187:3653:x1871938510657920:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18291us (18510us total) trans 4295070560 rc 0/0 00010000:00010000:2.0:1785220581.704651:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.704654:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.704654:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284305 00010000:00010000:2.0:1785220581.704661:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704667:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.704671:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704677:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a4cdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704677:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704689:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704691:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.704693:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.704695:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.704698:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704705:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704715:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.704717:0:13869:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095084431/0xf62499ab9278861c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bc89 expref: 332 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.704728:0:13869:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510659072:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.704742:0:13869:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704750:0:13869:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704769:0:13869:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704778:0:13869:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704785:0:13869:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704797:0:13869:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.704800:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d935a2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1187:3653:x1871938510658048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18629us (18745us total) trans 4295070561 rc 0/0 00000100:00100000:0.0:1785220581.704807:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284304 00000100:00100000:2.0:1785220581.704861:0:13869:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1119:3653:x1871938510658944:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12555us (12610us total) trans 4295069975 rc 0/0 00000100:00100000:2.0:1785220581.704867:0:13869:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284308 00010000:00010000:2.0:1785220581.704881:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.704885:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704892:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.704895:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704899:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.704904:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704904:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704911:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.704914:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704916:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704924:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4cd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704924:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704933:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704937:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.704939:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.704940:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.704941:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a4cf2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7215:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704953:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.704954:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.704956:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.704961:0:14257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e1f7564c/0xf62499ab9278871f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd2a expref: 331 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.704964:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.704972:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510659328:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.704982:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.704990:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.704997:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510660608:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.704998:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.705005:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.705012:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.705025:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.705030:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.705058:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510660608 00000100:00100000:0.0:1785220581.705064:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510660608 00000100:00100000:0.0:1785220581.705378:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284314 00000100:00100000:0.0:1785220581.705388:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1120:3655:x1871938510660608:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.705416:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29208 count 1 00010000:00010000:0.0:1785220581.705436:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.705451:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000062bdd60d/0xf62499ab9278896b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be42 expref: 330 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.705459:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7218:0x0].0x0 since=28s v=(0 0) 00010000:00010000:0.0:1785220581.705462:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7218:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.705466:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.705475:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.705493:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000062bdd60d/0xf62499ab9278896b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be42 expref: 330 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.705505:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000062bdd60d/0xf62499ab9278896b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be42 expref: 330 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.705517:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510660864:192.168.201.51@tcp:104: 00000004:00080000:3.0:1785220581.706465:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb65180/2, rc 0, transno 4295069975 00000100:00100000:3.0:1785220581.706477:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510658944:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.706498:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601fb80/2, rc 0, transno 4295070560 00000100:00100000:3.0:1785220581.706510:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510657920:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.706521:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fdf80/2, rc 0, transno 4295070561 00000100:00100000:3.0:1785220581.706524:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510658048:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.706582:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb67480/2, rc 0, transno 4295069976 00000100:00100000:3.0:1785220581.706586:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510659200:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.706594:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93665500/2, rc 0, transno 4295070559 00000100:00100000:3.0:1785220581.706598:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93665500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510656256:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.706659:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510660992:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.706801:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.706855:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510660480:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.706879:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.706907:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510660736 00000100:00100000:3.0:1785220581.706915:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510660736 00000100:00100000:3.0:1785220581.706954:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510660992 00000100:00100000:3.0:1785220581.706957:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510660992 00000100:00100000:3.0:1785220581.706973:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510660480 00000100:00100000:3.0:1785220581.706976:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510660480 00000100:00100000:3.0:1785220581.706995:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284315 00000100:00100000:3.0:1785220581.707003:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1123:3653:x1871938510660736:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.707025:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29209 count 1 00010000:00010000:3.0:1785220581.707048:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707075:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007f1f8a95/0xf62499ab927889b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be81 expref: 330 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707111:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7219:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.707115:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7219:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.707120:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707145:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007f1f8a95/0xf62499ab927889b8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be81 expref: 330 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707156:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007f1f8a95/0xf62499ab927889b8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be81 expref: 330 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.707167:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510661248:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.707563:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.707590:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284316 00000100:00100000:3.0:1785220581.707600:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1123:3653:x1871938510660992:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.707615:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29210 count 1 00010000:00010000:3.0:1785220581.707628:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707640:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009e079c93/0xf62499ab92788ad0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf1b expref: 330 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707651:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x721a:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.707654:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.707659:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707679:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009e079c93/0xf62499ab92788ad0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf1b expref: 330 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707688:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009e079c93/0xf62499ab92788ad0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf1b expref: 330 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.707697:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510661376:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.707722:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.707739:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284317 00000100:00100000:3.0:1785220581.707747:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1123:3654:x1871938510660480:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.707958:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29207 count 1 00010000:00010000:3.0:1785220581.707972:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707985:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000061a78a43/0xf62499ab9278885a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd8c expref: 330 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.707991:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7217:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.707994:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7217:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.707998:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.708017:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000061a78a43/0xf62499ab9278885a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd8c expref: 330 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.708027:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000061a78a43/0xf62499ab9278885a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd8c expref: 330 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.708038:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510661504:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.708065:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.708202:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510661120:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.709048:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.709119:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510661120 00000100:00100000:3.0:1785220581.709179:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510661120 00000100:00100000:3.0:1785220581.709307:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284318 00000100:00100000:3.0:1785220581.709319:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1124:3654:x1871938510661120:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.709340:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29211 count 1 00010000:00010000:3.0:1785220581.709613:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.709686:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000034ad2601/0xf62499ab92788b86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf76 expref: 330 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.709695:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x721b:0x0].0x0 since=28s v=(0 0) 00010000:00010000:3.0:1785220581.709702:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.709707:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.709742:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000034ad2601/0xf62499ab92788b86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf76 expref: 330 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.709756:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000034ad2601/0xf62499ab92788b86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf76 expref: 330 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.709782:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99066300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510661632:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.709819:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.710359:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb66d80/2, rc 0, transno 4295069974 00000100:00100000:3.0:1785220581.710376:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb66d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510658688:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.710425:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220581.710430:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99067480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510661760:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.710463:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.710655:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1124:3652:x1871938510659200:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17376us (17436us total) trans 4295069976 rc 0/0 00000100:00100000:2.0:1785220581.710665:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284309 00000100:00100000:2.0:1785220581.710695:0:14175:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510661760 00000100:00100000:2.0:1785220581.710701:0:14175:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510661760 00000100:00100000:2.0:1785220581.710743:0:14175:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284319 00000100:00100000:2.0:1785220581.710750:0:14175:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1124:3654:x1871938510661760:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.710775:0:14175:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29212 count 1 00010000:00010000:2.0:1785220581.710794:0:14175:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.710810:0:14175:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000025ca62fa/0xf62499ab92788cd6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bffb expref: 330 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.710818:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x721c:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.710823:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.710827:0:14175:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.710850:0:14175:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000025ca62fa/0xf62499ab92788cd6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bffb expref: 330 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.710860:0:14175:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000025ca62fa/0xf62499ab92788cd6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bffb expref: 330 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.710870:0:14175:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510661888:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.710899:0:14175:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.710909:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d263c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.710934:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90143480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510660096:192.168.201.51@tcp:104: 00000004:00080000:0.0:1785220581.710935:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e06d80/2, rc 0, transno 4295069973 00000100:00100000:0.0:1785220581.710945:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510655872:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.710955:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.710973:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.710980:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.710980:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:2.0:1785220581.710987:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:0.0:1785220581.710988:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510662016:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.710991:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.711001:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.711011:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.711025:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.711033:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a4cf9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7216:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.711036:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510662016 00000100:00100000:0.0:1785220581.711039:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510662016 00010000:00010000:2.0:1785220581.711053:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.711055:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.711057:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.711062:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284320 00000100:00100000:0.0:1785220581.711068:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1125:3653:x1871938510662016:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.711070:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00080000:0.0:1785220581.711080:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29213 count 1 00000100:00100000:1.0:1785220581.711244:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.711262:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938304 00000100:00100000:1.0:1785220581.711265:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938304 00010000:00010000:1.0:1785220581.711287:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000063d132a0/0xf62499ab9278b3aa lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2ee expref: 293 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.711312:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100243 00000100:00100000:1.0:1785220581.711318:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+293:2404:x1871938519938304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.711328:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.711334:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000063d132a0/0xf62499ab9278b3aa lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2ee expref: 293 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711344:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000063d132a0/0xf62499ab9278b3aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2ee expref: 293 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.711346:0:14170:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510659712:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.711361:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063d132a0/0xf62499ab9278b3aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2ee expref: 293 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.711362:0:14170:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711380:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7246:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.711383:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7246:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.711388:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711397:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711413:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.711431:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+292:2404:x1871938519938304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (188us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.711433:0:14170:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.711438:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100243 00010000:00010000:3.0:1785220581.711442:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.711456:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.711463:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.711470:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.711479:0:14170:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.711486:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938176 00000100:00100000:1.0:1785220581.711489:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938176 00010000:00010000:1.0:1785220581.711501:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bafaa249/0xf62499ab9278b4f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d381 expref: 292 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.711521:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100244 00000100:00100000:1.0:1785220581.711527:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+292:2407:x1871938519938176:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.711533:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.711538:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bafaa249/0xf62499ab9278b4f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d381 expref: 292 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711546:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bafaa249/0xf62499ab9278b4f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d381 expref: 292 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711557:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bafaa249/0xf62499ab9278b4f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d381 expref: 292 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711566:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7247:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.711569:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7247:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.711572:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711579:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.711594:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.711607:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+291:2407:x1871938519938176:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 80us (136us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.711614:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100244 00000100:00100000:3.0:1785220581.711624:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+1187:3654:x1871938510659456:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12237us (12609us total) trans 4295070562 rc 0/0 00000100:00100000:3.0:1785220581.711631:0:14170:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284310 00000004:00080000:3.0:1785220581.711661:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb20700/2, rc 0, transno 4295070562 00000100:00100000:3.0:1785220581.711666:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510659456:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.711768:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510662144:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.711797:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.711958:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.711968:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.711976:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.711983:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.711999:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.712082:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1187:3654:x1871938510659584:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11538us (12762us total) trans 4295070563 rc 0/0 00000100:00100000:2.0:1785220581.712180:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284311 00000100:00100000:2.0:1785220581.712195:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510662144 00000100:00100000:2.0:1785220581.712199:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510662144 00000100:00100000:2.0:1785220581.712290:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284321 00000100:00100000:2.0:1785220581.712297:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1187:3653:x1871938510662144:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.712301:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.712310:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29258 count 1 00010000:00010000:0.0:1785220581.712314:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000f51b32b/0xf62499ab92788dee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c080 expref: 330 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712321:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x721d:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.712322:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712324:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.712328:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712332:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d74b69f8/0xf62499ab9278b6dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4a7 expref: 290 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712339:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x724a:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.712341:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.712345:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712350:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000f51b32b/0xf62499ab92788dee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c080 expref: 330 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712359:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d74b69f8/0xf62499ab9278b6dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4a7 expref: 290 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712360:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000f51b32b/0xf62499ab92788dee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c080 expref: 330 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712369:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d74b69f8/0xf62499ab9278b6dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4a7 expref: 290 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.712372:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510662656:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.712378:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510662784:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.712408:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.712424:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.712429:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.712434:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.712438:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.712442:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712450:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.712455:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90143480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510662272:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.712463:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712470:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a4d00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712493:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.712497:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.712504:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.712504:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.712507:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712514:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712524:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.712526:0:14250:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510662272 00000100:00100000:2.0:1785220581.712529:0:14250:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510662272 00010000:00010000:0.0:1785220581.712531:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a4d07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.712544:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.712546:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.712548:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.712551:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284322 00010000:00010000:0.0:1785220581.712554:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.712557:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1188:3654:x1871938510662272:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.712568:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29259 count 1 00000100:00100000:0.0:1785220581.712574:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510660224:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.712579:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712590:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c606daf6/0xf62499ab9278b7af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4f4 expref: 290 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712596:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x724b:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.712596:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712599:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.712603:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712622:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c606daf6/0xf62499ab9278b7af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4f4 expref: 290 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.712622:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510662528:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.712632:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c606daf6/0xf62499ab9278b7af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4f4 expref: 290 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.712641:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510662912:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.712646:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.712656:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.712671:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510662528 00000100:00100000:2.0:1785220581.712674:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510662528 00000004:00080000:0.0:1785220581.712687:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fc000/2, rc 0, transno 4295070558 00000100:00100000:0.0:1785220581.712697:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510655488:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.712698:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284323 00000100:00100000:2.0:1785220581.712704:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1189:3652:x1871938510662528:12345-0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.712711:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97842d80/2, rc 0, transno 4295069971 00002000:00080000:2.0:1785220581.712714:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29261 count 1 00000100:00100000:0.0:1785220581.712718:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510656000:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.712726:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712737:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb152e15/0xf62499ab9278b97d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5db expref: 290 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712743:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x724d:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.712746:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.712750:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.712751:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220581.712756:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510663040:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.712780:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fb152e15/0xf62499ab9278b97d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5db expref: 290 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.712791:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fb152e15/0xf62499ab9278b97d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5db expref: 290 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.712792:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.712800:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510663168:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.712812:0:14206:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510663040 00000100:00100000:0.0:1785220581.712815:0:14206:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510663040 00000100:00100000:2.0:1785220581.712818:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.712837:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284324 00000100:00100000:1.0:1785220581.712974:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.713018:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1126:3652:x1871938510663040:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.713030:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29214 count 1 00010000:00010000:0.0:1785220581.713043:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713054:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006cf642b2/0xf62499ab92788ef1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0fe expref: 330 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.713061:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb23b80/2, rc 0, transno 4295070563 00010000:00010000:0.0:1785220581.713061:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x721e:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.713064:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721e:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220581.713068:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510659584:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.713069:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.713197:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510663296:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.713240:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006cf642b2/0xf62499ab92788ef1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0fe expref: 330 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713251:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006cf642b2/0xf62499ab92788ef1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0fe expref: 330 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.713261:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510663552:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.713288:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26a80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.713296:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.713313:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220581.713315:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510662400:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.713318:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510663424:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.713335:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.713361:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510663296 00000100:00100000:2.0:1785220581.713365:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.713365:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510663296 00000100:00100000:0.0:1785220581.713392:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510663424 00000100:00100000:2.0:1785220581.713393:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510662400 00000100:00100000:2.0:1785220581.713395:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510662400 00000100:00100000:0.0:1785220581.713395:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510663424 00000100:00100000:0.0:1785220581.713413:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284325 00000100:00100000:2.0:1785220581.713419:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284326 00000100:00100000:0.0:1785220581.713419:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1192:3654:x1871938510663296:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.713423:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1192:3654:x1871938510663424:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.713431:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29262 count 1 00002000:00080000:2.0:1785220581.713432:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29263 count 1 00010000:00010000:2.0:1785220581.713443:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713443:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.713456:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd89558e/0xf62499ab9278bb91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6c2 expref: 290 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713456:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003a2aa8a5/0xf62499ab9278bacd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d660 expref: 290 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.713464:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x724f:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.713464:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x724e:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.713468:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.713468:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.713472:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713472:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.713489:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dd89558e/0xf62499ab9278bb91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6c2 expref: 290 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713490:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003a2aa8a5/0xf62499ab9278bacd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d660 expref: 290 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.713500:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dd89558e/0xf62499ab9278bb91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6c2 expref: 290 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713500:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003a2aa8a5/0xf62499ab9278bacd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d660 expref: 290 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.713512:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510663680:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.713512:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510663808:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.713538:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.713560:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284327 00000100:00100000:3.0:1785220581.713570:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1192:3655:x1871938510662400:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.713582:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29260 count 1 00010000:00010000:3.0:1785220581.713594:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.713605:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d20af132/0xf62499ab9278b89d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d564 expref: 290 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.713612:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x724c:0x0].0x0 since=25s v=(0 0) 00010000:00010000:3.0:1785220581.713615:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.713618:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.713634:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938432 00010000:00010000:3.0:1785220581.713637:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d20af132/0xf62499ab9278b89d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d564 expref: 290 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.713638:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938432 00010000:00010000:3.0:1785220581.713646:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d20af132/0xf62499ab9278b89d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d564 expref: 291 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.713655:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510663936:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.713655:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d99f2dff/0xf62499ab9278b60b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d429 expref: 291 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.713673:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.713710:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.713718:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100245 00000100:00100000:1.0:1785220581.713726:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+291:2406:x1871938519938432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.713735:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.713740:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d99f2dff/0xf62499ab9278b60b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d429 expref: 291 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.713750:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d99f2dff/0xf62499ab9278b60b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d429 expref: 291 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.713912:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d99f2dff/0xf62499ab9278b60b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d429 expref: 291 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.713921:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7249:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.713923:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7249:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.713927:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.713932:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.713949:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.713968:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2406:x1871938519938432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 243us (995us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.713971:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.713975:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100245 00010000:00010000:0.0:1785220581.713980:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713989:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.713996:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.714045:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.714550:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1193:3653:x1871938510659968:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13853us (14799us total) trans 4295070564 rc 0/0 00000100:00100000:0.0:1785220581.714561:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284312 00010000:00010000:2.0:1785220581.714647:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.714653:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.714662:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.714666:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.714675:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.714688:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.714696:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a4d0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.714710:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.714711:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.714713:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.714757:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb22d80/2, rc 0, transno 4295070564 00000100:00100000:3.0:1785220581.714811:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510659968:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.714909:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510664064:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.714939:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.715000:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510664064 00000100:00100000:3.0:1785220581.715006:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510664064 00000100:00100000:3.0:1785220581.715045:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284328 00000100:00100000:3.0:1785220581.715053:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1193:3652:x1871938510664064:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.715071:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29264 count 1 00010000:00010000:3.0:1785220581.715113:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.715130:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b1e0a43f/0xf62499ab9278bcb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d739 expref: 289 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.715138:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7250:0x0].0x0 since=25s v=(0 0) 00010000:00010000:3.0:1785220581.715142:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7250:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.715145:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.715167:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b1e0a43f/0xf62499ab9278bcb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d739 expref: 289 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.715178:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b1e0a43f/0xf62499ab9278bcb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d739 expref: 289 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.715189:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510664192:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.715239:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.722333:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722368:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722386:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722401:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722416:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722430:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722468:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722494:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722512:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722529:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.722555:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938688 00000100:00100000:1.0:1785220581.722564:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938688 00010000:00010000:1.0:1785220581.722602:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007f1f8a95/0xf62499ab927889b8 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be81 expref: 331 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722633:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938944 00000100:00100000:1.0:1785220581.722636:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938944 00010000:00010000:1.0:1785220581.722648:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000061a78a43/0xf62499ab9278885a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd8c expref: 332 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722664:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939200 00000100:00100000:1.0:1785220581.722666:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939200 00010000:00010000:1.0:1785220581.722677:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000062bdd60d/0xf62499ab9278896b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be42 expref: 333 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722692:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938560 00000100:00100000:1.0:1785220581.722694:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938560 00010000:00010000:1.0:1785220581.722707:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c6a41fed/0xf62499ab9278b55c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3b9 expref: 290 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722731:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939456 00000100:00100000:1.0:1785220581.722734:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939456 00010000:00010000:1.0:1785220581.722746:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000f51b32b/0xf62499ab92788dee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c080 expref: 334 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722775:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519938816 00000100:00100000:1.0:1785220581.722778:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519938816 00010000:00010000:1.0:1785220581.722791:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009e079c93/0xf62499ab92788ad0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf1b expref: 335 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722802:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939712 00000100:00100000:1.0:1785220581.722804:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939712 00010000:00010000:1.0:1785220581.722815:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c606daf6/0xf62499ab9278b7af lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4f4 expref: 291 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722825:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939072 00000100:00100000:1.0:1785220581.722827:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939072 00010000:00010000:1.0:1785220581.722847:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000034ad2601/0xf62499ab92788b86 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf76 expref: 336 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722859:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939968 00000100:00100000:1.0:1785220581.722862:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939968 00010000:00010000:1.0:1785220581.722872:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006cf642b2/0xf62499ab92788ef1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0fe expref: 337 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722882:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939328 00000100:00100000:1.0:1785220581.722885:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939328 00010000:00010000:1.0:1785220581.722896:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000025ca62fa/0xf62499ab92788cd6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bffb expref: 338 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.722911:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100246 00000100:00100000:1.0:1785220581.722920:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e11450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+338:2406:x1871938519938688:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.722934:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.722939:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007f1f8a95/0xf62499ab927889b8 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be81 expref: 338 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.722951:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007f1f8a95/0xf62499ab927889b8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be81 expref: 338 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.722977:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7219:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.722982:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7219:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.722988:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.722998:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723021:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723074:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e11450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+338:2406:x1871938519938688:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 154us (754us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723082:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100246 00000100:00100000:1.0:1785220581.723157:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100247 00000100:00100000:1.0:1785220581.723165:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2406:x1871938519938944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723172:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723176:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000061a78a43/0xf62499ab9278885a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd8c expref: 337 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723185:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000061a78a43/0xf62499ab9278885a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd8c expref: 337 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723198:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7217:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723201:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7217:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723206:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723213:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723224:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723242:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+337:2406:x1871938519938944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723249:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100247 00000100:00100000:1.0:1785220581.723262:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100248 00000100:00100000:1.0:1785220581.723268:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+336:2406:x1871938519939200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723275:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723280:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000062bdd60d/0xf62499ab9278896b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be42 expref: 336 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723296:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000062bdd60d/0xf62499ab9278896b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be42 expref: 336 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723306:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7218:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723308:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7218:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723311:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723315:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723323:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723336:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+336:2406:x1871938519939200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 69us (949us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723342:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100248 00000100:00100000:1.0:1785220581.723355:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100249 00000100:00100000:1.0:1785220581.723362:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+291:2407:x1871938519938560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723369:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723374:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c6a41fed/0xf62499ab9278b55c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3b9 expref: 291 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723385:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c6a41fed/0xf62499ab9278b55c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3b9 expref: 291 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723404:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7248:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723407:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7248:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723410:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723417:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723428:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723445:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+291:2407:x1871938519938560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (1042us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723450:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100249 00000100:00100000:1.0:1785220581.723463:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100250 00000100:00100000:1.0:1785220581.723470:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+335:2406:x1871938519939456:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723477:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723482:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000f51b32b/0xf62499ab92788dee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c080 expref: 335 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723491:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000f51b32b/0xf62499ab92788dee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c080 expref: 335 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723503:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723506:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723510:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723519:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723531:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723547:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+335:2406:x1871938519939456:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (1130us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723553:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100250 00000100:00100000:1.0:1785220581.723566:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100251 00000100:00100000:1.0:1785220581.723572:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+334:2407:x1871938519938816:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723579:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723584:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009e079c93/0xf62499ab92788ad0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf1b expref: 334 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723593:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009e079c93/0xf62499ab92788ad0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf1b expref: 334 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723605:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723608:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723612:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723619:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723635:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723653:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+334:2407:x1871938519938816:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 81us (1221us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723659:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100251 00000100:00100000:1.0:1785220581.723671:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100252 00000100:00100000:1.0:1785220581.723678:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2406:x1871938519939712:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723685:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723690:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c606daf6/0xf62499ab9278b7af lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4f4 expref: 290 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723700:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c606daf6/0xf62499ab9278b7af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4f4 expref: 290 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723712:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723715:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723719:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723726:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723737:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723755:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2406:x1871938519939712:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (1286us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723778:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100252 00000100:00100000:1.0:1785220581.723790:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100253 00000100:00100000:1.0:1785220581.723796:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce7a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+333:2407:x1871938519939072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723803:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723808:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000034ad2601/0xf62499ab92788b86 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf76 expref: 333 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723817:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000034ad2601/0xf62499ab92788b86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf76 expref: 333 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723830:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723833:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723837:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723844:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723855:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723874:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce7a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+333:2407:x1871938519939072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (1379us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723879:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100253 00000100:00100000:1.0:1785220581.723907:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100254 00000100:00100000:1.0:1785220581.723913:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+332:2406:x1871938519939968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.723922:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.723926:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006cf642b2/0xf62499ab92788ef1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0fe expref: 332 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723935:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006cf642b2/0xf62499ab92788ef1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0fe expref: 332 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723946:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723949:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.723953:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723960:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.723970:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.723983:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+332:2406:x1871938519939968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 70us (1471us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.723989:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100254 00000100:00100000:1.0:1785220581.724001:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100255 00000100:00100000:1.0:1785220581.724006:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+331:2407:x1871938519939328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.724013:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.724017:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000025ca62fa/0xf62499ab92788cd6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bffb expref: 331 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724032:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000025ca62fa/0xf62499ab92788cd6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bffb expref: 331 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724043:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724046:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724051:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724057:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724068:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.724081:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+331:2407:x1871938519939328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (1552us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.724113:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100255 00000100:00100000:1.0:1785220581.724187:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.724215:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940224 00000100:00100000:1.0:1785220581.724221:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940224 00010000:00010000:1.0:1785220581.724239:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d20af132/0xf62499ab9278b89d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d564 expref: 290 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.724261:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100256 00000100:00100000:1.0:1785220581.724267:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2406:x1871938519940224:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.724275:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.724279:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d20af132/0xf62499ab9278b89d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d564 expref: 290 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724289:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d20af132/0xf62499ab9278b89d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d564 expref: 290 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724300:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724303:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724308:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724315:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724327:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.724340:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2406:x1871938519940224:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 74us (153us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.724347:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100256 00000100:00100000:1.0:1785220581.724404:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.724416:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939584 00000100:00100000:1.0:1785220581.724418:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939584 00010000:00010000:1.0:1785220581.724428:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d74b69f8/0xf62499ab9278b6dd lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4a7 expref: 290 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.724441:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100257 00000100:00100000:1.0:1785220581.724444:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2407:x1871938519939584:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.724448:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.724451:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d74b69f8/0xf62499ab9278b6dd lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4a7 expref: 290 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724457:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d74b69f8/0xf62499ab9278b6dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4a7 expref: 290 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724465:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724467:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724470:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724475:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724482:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.724491:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2407:x1871938519939584:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 46us (87us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.724494:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100257 00000100:00100000:1.0:1785220581.724522:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.724533:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519939840 00000100:00100000:1.0:1785220581.724535:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519939840 00010000:00010000:1.0:1785220581.724543:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fb152e15/0xf62499ab9278b97d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5db expref: 290 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.724555:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100258 00000100:00100000:1.0:1785220581.724559:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2407:x1871938519939840:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.724563:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.724566:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fb152e15/0xf62499ab9278b97d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5db expref: 290 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724572:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fb152e15/0xf62499ab9278b97d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5db expref: 290 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724580:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724582:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724585:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724590:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724597:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.724604:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2407:x1871938519939840:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 45us (82us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.724608:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100258 00000100:00100000:1.0:1785220581.724636:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.724645:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940096 00000100:00100000:1.0:1785220581.724647:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940096 00010000:00010000:1.0:1785220581.724655:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dd89558e/0xf62499ab9278bb91 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6c2 expref: 290 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.724668:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100259 00000100:00100000:1.0:1785220581.724672:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2407:x1871938519940096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.724676:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.724680:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dd89558e/0xf62499ab9278bb91 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6c2 expref: 290 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724686:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dd89558e/0xf62499ab9278bb91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6c2 expref: 290 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724693:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724695:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.724698:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724703:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.724710:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.724717:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2407:x1871938519940096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 45us (82us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.724721:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100259 00010000:00010000:2.0:1785220581.725160:0:14175:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025ca62fa/0xf62499ab92788cd6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bffb expref: 330 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.725183:0:14175:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510661888:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.725215:0:14175:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725227:0:14175:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725234:0:14175:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725245:0:14175:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725254:0:14175:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725273:0:14175:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.725443:0:14175:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1127:3654:x1871938510661760:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14693us (14978us total) trans 4295069977 rc 0/0 00000100:00100000:2.0:1785220581.725451:0:14175:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284319 00010000:00010000:2.0:1785220581.725486:0:14174:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb152e15/0xf62499ab9278b97d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5db expref: 289 pid: 13837 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.725497:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510663168:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.725509:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725517:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725523:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725532:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725538:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725555:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.725644:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1194:3652:x1871938510662528:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12939us (12996us total) trans 4295070565 rc 0/0 00000100:00100000:2.0:1785220581.725652:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284323 00010000:00010000:2.0:1785220581.725682:0:14177:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd89558e/0xf62499ab9278bb91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6c2 expref: 288 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.725693:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510663680:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.725705:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725719:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725726:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725733:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725741:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725752:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.725837:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1194:3654:x1871938510663424:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12412us (12500us total) trans 4295070566 rc 0/0 00000100:00100000:2.0:1785220581.725844:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284326 00010000:00010000:2.0:1785220581.725864:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.725870:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725879:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.725883:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725893:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725909:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725917:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a4d3f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725934:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.725938:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725945:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.725949:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725956:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725967:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725974:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a4d5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725987:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.725990:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.725997:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.726000:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726009:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726018:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726024:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a4d70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726034:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726036:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726037:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726059:0:14160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d74b69f8/0xf62499ab9278b6dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4a7 expref: 287 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.726068:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510662784:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.726079:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726114:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726120:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726127:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726134:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726148:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.726222:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1194:3653:x1871938510662144:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13925us (14423us total) trans 4295070567 rc 0/0 00000100:00100000:2.0:1785220581.726228:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284321 00010000:00010000:2.0:1785220581.726246:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.726250:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726257:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.726260:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726268:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726279:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726287:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a4d4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726302:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726304:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726305:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726327:0:14250:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c606daf6/0xf62499ab9278b7af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4f4 expref: 286 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.726336:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510662912:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.726347:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726355:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726362:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726370:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726377:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726394:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.726462:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1194:3654:x1871938510662272:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13905us (13957us total) trans 4295070568 rc 0/0 00000100:00100000:2.0:1785220581.726469:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284322 00010000:00010000:2.0:1785220581.726485:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.726489:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726496:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.726498:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726506:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726517:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726524:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a4d54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726537:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726538:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726540:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.726545:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.726577:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb67100/2, rc 0, transno 4295070567 00000100:00100000:2.0:1785220581.726589:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510662144:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.726657:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510664320:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.726689:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.726747:0:14267:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c6a41fed/0xf62499ab9278b55c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3b9 expref: 285 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.726772:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510660352:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.726784:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726791:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726797:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726803:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726809:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.726819:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.726890:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1194:3653:x1871938510659840:12345-0@lo:6:osp-syn-0-0.0 Request processed in 26110us (26441us total) trans 4295070569 rc 0/0 00000100:00100000:2.0:1785220581.726897:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284313 00000100:00100000:2.0:1785220581.726910:0:14267:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510664320 00000100:00100000:2.0:1785220581.726915:0:14267:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510664320 00000100:00100000:2.0:1785220581.726945:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284329 00000100:00100000:2.0:1785220581.726953:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1194:3653:x1871938510664320:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.726978:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29265 count 1 00010000:00010000:2.0:1785220581.726998:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727012:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b300e288/0xf62499ab9278bd2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7b7 expref: 284 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727019:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7251:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.727024:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7251:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.727027:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727052:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b300e288/0xf62499ab9278bd2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7b7 expref: 284 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727064:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b300e288/0xf62499ab9278bd2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7b7 expref: 284 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.727075:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510664448:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.727140:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.727151:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.727155:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727162:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.727165:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727172:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727183:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727191:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a4d15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727206:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.727208:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.727209:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.727227:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb21500/2, rc 0, transno 4295070569 00000100:00100000:2.0:1785220581.727235:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510659840:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.727275:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510664576:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.727303:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.727326:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7e680/2, rc 0, transno 4295070566 00000100:00100000:2.0:1785220581.727334:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510663424:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.727365:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90143480/2, rc 0, transno 4295070568 00000100:00100000:2.0:1785220581.727373:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90143480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510662272:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.727397:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.727403:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510664832:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.727427:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.727451:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510664576 00000100:00100000:2.0:1785220581.727454:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510664576 00000100:00100000:2.0:1785220581.727479:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510664832 00000100:00100000:2.0:1785220581.727482:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510664832 00000100:00100000:2.0:1785220581.727501:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284330 00000100:00100000:2.0:1785220581.727507:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1196:3654:x1871938510664576:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.727520:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29266 count 1 00010000:00010000:2.0:1785220581.727532:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727544:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b5ed905a/0xf62499ab9278be5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d819 expref: 284 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727550:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7252:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.727554:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7252:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.727557:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727574:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b5ed905a/0xf62499ab9278be5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d819 expref: 284 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727584:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b5ed905a/0xf62499ab9278be5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d819 expref: 284 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.727594:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510664960:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.727612:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.727648:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284331 00000100:00100000:2.0:1785220581.727656:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1196:3653:x1871938510664832:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.727668:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29268 count 1 00010000:00010000:2.0:1785220581.727680:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727690:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000bc62747/0xf62499ab9278c061 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8f2 expref: 284 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727695:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7254:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.727697:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7254:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.727699:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727710:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000bc62747/0xf62499ab9278c061 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8f2 expref: 284 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.727716:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000bc62747/0xf62499ab9278c061 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8f2 expref: 284 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.727723:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510665088:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.727737:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.727754:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99067480/2, rc 0, transno 4295069977 00000100:00100000:2.0:1785220581.727779:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99067480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510661760:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.727820:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510665216:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.727844:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.727884:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.727889:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510664704:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.727913:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.727932:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510665216 00000100:00100000:2.0:1785220581.727936:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510665216 00000100:00100000:2.0:1785220581.727962:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510664704 00000100:00100000:2.0:1785220581.727964:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510664704 00000100:00100000:2.0:1785220581.727982:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284332 00000100:00100000:2.0:1785220581.727989:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1127:3653:x1871938510665216:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.728000:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29215 count 1 00010000:00010000:2.0:1785220581.728016:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728028:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000094899ec8/0xf62499ab9278901e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c191 expref: 329 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728035:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x721f:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.728037:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.728040:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728058:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000094899ec8/0xf62499ab9278901e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c191 expref: 329 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728068:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000094899ec8/0xf62499ab9278901e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c191 expref: 329 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.728077:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510665344:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.728132:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.728146:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284333 00000100:00100000:2.0:1785220581.728153:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b09f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1197:3654:x1871938510664704:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.728165:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29267 count 1 00010000:00010000:2.0:1785220581.728177:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728189:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000026e3a62d/0xf62499ab9278bf42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d890 expref: 284 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728196:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7253:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.728199:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7253:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.728203:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728222:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000026e3a62d/0xf62499ab9278bf42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d890 expref: 284 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.728231:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000026e3a62d/0xf62499ab9278bf42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d890 expref: 284 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.728247:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510665472:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.728266:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.728437:0:14207:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007f1f8a95/0xf62499ab927889b8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be81 expref: 329 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.728452:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510661248:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.728469:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728478:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728484:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728492:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728499:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728528:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.728677:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1128:3653:x1871938510660736:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21674us (23201us total) trans 4295069978 rc 0/0 00000100:00100000:0.0:1785220581.728690:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284315 00000100:00100000:0.0:1785220581.728740:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510663808:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.728756:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728835:0:14206:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006cf642b2/0xf62499ab92788ef1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c0fe expref: 328 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.728846:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510663552:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.728863:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728871:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728877:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728885:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728891:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.728910:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.728992:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1128:3652:x1871938510663040:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15980us (16199us total) trans 4295069979 rc 0/0 00000100:00100000:0.0:1785220581.728999:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284324 00010000:00010000:0.0:1785220581.729017:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.729024:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729031:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.729035:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729044:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729061:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729068:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4d23 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7219:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729129:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.729134:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729145:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.729148:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729155:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729166:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729172:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4d62 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729185:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.729187:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.729189:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.729206:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.729228:0:14256:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f51b32b/0xf62499ab92788dee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c080 expref: 327 pid: 14205 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.729239:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510662656:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.729251:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729259:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729266:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729273:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729280:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729291:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.729363:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1128:3653:x1871938510662016:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18294us (18350us total) trans 4295069980 rc 0/0 00000100:00100000:0.0:1785220581.729370:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284320 00010000:00010000:0.0:1785220581.729410:0:13909:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062bdd60d/0xf62499ab9278896b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7be42 expref: 326 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.729421:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510660864:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.729432:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729440:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729448:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729455:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729461:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729473:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.729543:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1128:3655:x1871938510660608:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24157us (24513us total) trans 4295069981 rc 0/0 00000100:00100000:0.0:1785220581.729549:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284314 00010000:00010000:0.0:1785220581.729567:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.729572:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729579:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.729582:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729590:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729601:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729608:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a4d46 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729625:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.729629:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729636:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.729639:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729646:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729656:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729663:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a4d1c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7218:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.729675:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.729677:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.729679:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.729709:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083d500/2, rc 0, transno 4295069979 00000100:00100000:0.0:1785220581.729718:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510663040:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.729730:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbed80/2, rc 0, transno 4295070565 00000100:00100000:0.0:1785220581.729740:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510662528:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.732888:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.732905:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.732974:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083c000/2, rc 0, transno 4295069980 00000100:00100000:2.0:1785220581.732988:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510662016:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.733005:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be7800/2, rc 0, transno 4295069978 00000100:00100000:2.0:1785220581.733009:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510660736:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.733322:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.733342:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94913100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510665856:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.733409:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.733467:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94912d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510665984:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.733503:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.733538:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510665856 00000100:00100000:2.0:1785220581.733544:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510665856 00000100:00100000:2.0:1785220581.733585:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510665984 00000100:00100000:2.0:1785220581.733588:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510665984 00000100:00100000:2.0:1785220581.733613:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284334 00000100:00100000:2.0:1785220581.733622:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1129:3653:x1871938510665856:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.733656:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29217 count 1 00010000:00010000:2.0:1785220581.733685:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.733718:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004786e31a/0xf62499ab927891d7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c278 expref: 325 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.733729:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7221:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.733733:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7221:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.733738:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.733751:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.733788:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284335 00000100:00100000:1.0:1785220581.733799:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94912300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1129:3653:x1871938510665984:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.733811:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29218 count 1 00010000:00010000:1.0:1785220581.733897:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.733912:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002d75790c/0xf62499ab92789271 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2e1 expref: 325 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.733925:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7222:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.733928:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7222:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.733931:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.733953:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002d75790c/0xf62499ab92789271 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2e1 expref: 325 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.733966:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002d75790c/0xf62499ab92789271 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2e1 expref: 325 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.733978:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510666112:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.734026:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.734044:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940352 00000100:00100000:1.0:1785220581.734048:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940352 00010000:00010000:1.0:1785220581.734072:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003a2aa8a5/0xf62499ab9278bacd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d660 expref: 285 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.734327:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004786e31a/0xf62499ab927891d7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c278 expref: 325 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.734337:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004786e31a/0xf62499ab927891d7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c278 expref: 325 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.734348:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510666240:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.734373:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.734382:0:13867:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e079c93/0xf62499ab92788ad0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf1b expref: 325 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.734394:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510665600:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.734396:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510661376:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.734416:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.734422:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.734427:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.734434:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.734446:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:3.0:1785220581.734448:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.734451:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510665728:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.734456:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.734469:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.734471:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1785220581.734506:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991ad80/2, rc 0, transno 4295069981 00000100:00100000:2.0:1785220581.734514:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510660608:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.734600:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.734607:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.734615:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.734619:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.734627:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.734644:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.734651:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4d2a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.734668:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.734670:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.734672:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.734772:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1130:3653:x1871938510660992:12345-0@lo:6:osp-syn-1-0.0 Request processed in 27174us (27969us total) trans 4295069982 rc 0/0 00000100:00100000:3.0:1785220581.734780:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284316 00000100:00100000:1.0:1785220581.734948:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100260 00000100:00100000:1.0:1785220581.734957:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+285:2407:x1871938519940352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.734968:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.734974:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003a2aa8a5/0xf62499ab9278bacd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d660 expref: 285 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.734990:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003a2aa8a5/0xf62499ab9278bacd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d660 expref: 285 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.735005:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a2aa8a5/0xf62499ab9278bacd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d660 expref: 285 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.735016:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.735019:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x724e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.735025:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.735034:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.735341:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be6680/2, rc 0, transno 4295069982 00000100:00100000:2.0:1785220581.735349:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510660992:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.735387:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94910700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510666496:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.735417:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.735438:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.735443:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94913b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510666368:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.735461:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.735670:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.735696:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+284:2407:x1871938519940352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 739us (1943us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.735704:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100260 00000100:00100000:1.0:1785220581.735749:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510665600 00000100:00100000:1.0:1785220581.735755:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510665600 00000100:00100000:1.0:1785220581.735801:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510665728 00000100:00100000:1.0:1785220581.735804:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510665728 00000100:00100000:1.0:1785220581.735821:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510666496 00000100:00100000:1.0:1785220581.735824:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510666496 00000100:00100000:1.0:1785220581.735852:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510666368 00000100:00100000:1.0:1785220581.735854:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510666368 00000100:00100000:1.0:1785220581.735872:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284336 00000100:00100000:1.0:1785220581.735879:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1198:3654:x1871938510665600:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.735898:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29269 count 1 00010000:00010000:1.0:1785220581.735916:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.735931:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007c06d4ca/0xf62499ab9278c172 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d99a expref: 283 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.735938:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7255:0x0].0x0 since=25s v=(0 0) 00010000:00010000:1.0:1785220581.735941:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7255:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.735945:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.735965:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007c06d4ca/0xf62499ab9278c172 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d99a expref: 283 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.735975:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007c06d4ca/0xf62499ab9278c172 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d99a expref: 283 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.735986:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510666624:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.736015:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd480 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.736033:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284337 00000100:00100000:1.0:1785220581.736041:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1133:3654:x1871938510665728:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.736053:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29216 count 1 00010000:00010000:1.0:1785220581.736065:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736077:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000278d32e8/0xf62499ab927890b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c20f expref: 324 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736380:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7220:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.736385:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7220:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.736389:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736410:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000278d32e8/0xf62499ab927890b8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c20f expref: 324 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736418:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000278d32e8/0xf62499ab927890b8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c20f expref: 324 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.736429:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510666752:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.736450:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdb40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.736468:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284338 00000100:00100000:1.0:1785220581.736475:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1133:3655:x1871938510666496:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.736510:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29220 count 1 00010000:00010000:1.0:1785220581.736521:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736533:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d82dbf51/0xf62499ab927893c8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c389 expref: 324 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736545:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7224:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.736548:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7224:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.736553:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736570:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d82dbf51/0xf62499ab927893c8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c389 expref: 324 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736579:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d82dbf51/0xf62499ab927893c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c389 expref: 324 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.736589:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0001_UUID:14259:1871938510666880:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.736608:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdc00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.736625:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284339 00000100:00100000:1.0:1785220581.736632:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94912680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1133:3655:x1871938510666368:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.736643:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29219 count 1 00010000:00010000:1.0:1785220581.736654:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736665:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003791778a/0xf62499ab92789319 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c327 expref: 324 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736671:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7223:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.736674:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7223:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.736679:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736698:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003791778a/0xf62499ab92789319 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c327 expref: 324 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.736708:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003791778a/0xf62499ab92789319 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c327 expref: 324 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.736718:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510667008:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.736736:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.736960:0:14168:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000061a78a43/0xf62499ab9278885a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd8c expref: 324 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.736975:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510661504:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.737001:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737010:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737016:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737025:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737034:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737055:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.737243:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1133:3654:x1871938510660480:12345-0@lo:6:osp-syn-1-0.0 Request processed in 29497us (30362us total) trans 4295069983 rc 0/0 00000100:00100000:3.0:1785220581.737251:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284317 00010000:00010000:3.0:1785220581.737287:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034ad2601/0xf62499ab92788b86 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bf76 expref: 323 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.737294:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99919880/2, rc 0, transno 4295069983 00000100:00100000:3.0:1785220581.737297:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99066300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510661632:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.737305:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510660480:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.737309:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737316:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737323:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737331:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737337:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737350:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.737352:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.737361:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94913480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510667136:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.737389:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.737416:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:3.0:1785220581.737423:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1133:3654:x1871938510661120:12345-0@lo:6:osp-syn-1-0.0 Request processed in 28106us (28380us total) trans 4295069984 rc 0/0 00010000:00010000:2.0:1785220581.737423:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.737430:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284318 00010000:00010000:2.0:1785220581.737431:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.737436:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.737443:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510667136 00010000:00010000:2.0:1785220581.737444:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.737447:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510667136 00010000:00010000:2.0:1785220581.737461:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737469:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4d31 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7217:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.737478:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284340 00000100:00100000:3.0:1785220581.737483:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1133:3653:x1871938510667136:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.737497:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29221 count 1 00010000:00010000:2.0:1785220581.737506:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.737510:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737510:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737518:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.737521:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737523:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005a7f0339/0xf62499ab927894b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3dd expref: 322 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737529:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737531:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7225:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.737534:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7225:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.737538:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737539:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737547:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a4d38 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737557:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005a7f0339/0xf62499ab927894b6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3dd expref: 322 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737560:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.737562:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.737564:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.737567:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005a7f0339/0xf62499ab927894b6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3dd expref: 322 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.737584:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510667264:192.168.201.51@tcp:104: 00000004:00080000:2.0:1785220581.737589:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be6a00/2, rc 0, transno 4295069984 00000100:00100000:2.0:1785220581.737596:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510661120:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.737611:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.737641:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94912a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510667392:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.737654:0:14192:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d20af132/0xf62499ab9278b89d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d564 expref: 283 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.737663:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.737664:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510663936:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.737676:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737685:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.737689:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510667392 00010000:00010000:3.0:1785220581.737692:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.737692:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510667392 00010000:00010000:3.0:1785220581.737700:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.737707:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.737714:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284341 00010000:00010000:3.0:1785220581.737720:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.737720:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1134:3653:x1871938510667392:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.737730:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29222 count 1 00010000:00010000:2.0:1785220581.737743:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737754:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005cd1b3be/0xf62499ab927895a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c462 expref: 322 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737775:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7226:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.737778:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7226:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.737781:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737800:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005cd1b3be/0xf62499ab927895a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c462 expref: 322 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.737809:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005cd1b3be/0xf62499ab927895a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c462 expref: 322 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.737819:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510667520:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.737823:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1199:3655:x1871938510662400:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24256us (24457us total) trans 4295070570 rc 0/0 00000100:00100000:1.0:1785220581.738004:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.738009:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.738023:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220581.738023:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940480 00000100:00100000:1.0:1785220581.738027:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940480 00010000:00010000:2.0:1785220581.738028:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.738040:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.738044:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.738046:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b1e0a43f/0xf62499ab9278bcb0 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d739 expref: 283 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.738052:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.738064:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.738066:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100261 00010000:00010000:2.0:1785220581.738072:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4d77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.738073:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+283:2407:x1871938519940480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.738082:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.738101:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b1e0a43f/0xf62499ab9278bcb0 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d739 expref: 283 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.738112:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b1e0a43f/0xf62499ab9278bcb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d739 expref: 283 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.738134:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7250:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.738138:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7250:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.738143:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.738152:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.738166:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.738181:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+283:2407:x1871938519940480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (179us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.738188:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100261 00000100:00100000:3.0:1785220581.739127:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284327 00010000:00010000:3.0:1785220581.739181:0:14189:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1e0a43f/0xf62499ab9278bcb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d739 expref: 282 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.739196:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510664192:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.739217:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.739228:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.739236:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.739244:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.739252:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.739278:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.739465:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1199:3652:x1871938510664064:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24412us (24524us total) trans 4295070571 rc 0/0 00000100:00100000:3.0:1785220581.739472:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284328 00010000:00010000:2.0:1785220581.739498:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.739505:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.739512:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.739516:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.739524:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.739540:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.739547:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4d7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.739562:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.739564:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.739566:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.739592:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7dc00/2, rc 0, transno 4295070571 00000100:00100000:2.0:1785220581.739613:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510664064:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.739649:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbe680/2, rc 0, transno 4295070570 00000100:00100000:2.0:1785220581.739653:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510662400:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.740394:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.741272:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.741288:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.741299:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.741313:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.741342:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.741527:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1199:3654:x1871938510663296:12345-0@lo:6:osp-syn-0-0.0 Request processed in 28105us (28227us total) trans 4295070572 rc 0/0 00000100:00100000:0.0:1785220581.741536:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284325 00010000:00010000:2.0:1785220581.741682:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.741690:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.741700:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.741705:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.741716:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.741736:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.741751:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4d69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.741785:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.741787:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.741789:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.741837:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7ca80/2, rc 0, transno 4295070572 00000100:00100000:2.0:1785220581.741844:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510663296:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.743928:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.744713:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510667648:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.744793:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.745038:0:14223:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510667648 00000100:00100000:1.0:1785220581.745046:0:14223:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510667648 00000100:00100000:1.0:1785220581.745122:0:14223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284342 00000100:00100000:1.0:1785220581.745133:0:14223:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+1199:3654:x1871938510667648:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.745165:0:14223:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29270 count 1 00010000:00010000:1.0:1785220581.745194:0:14223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.745217:0:14223:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009faa73d4/0xf62499ab9278c2d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da26 expref: 281 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.745225:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7256:0x0].0x0 since=25s v=(0 0) 00010000:00010000:1.0:1785220581.745229:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7256:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.745234:0:14223:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.745261:0:14223:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009faa73d4/0xf62499ab9278c2d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da26 expref: 281 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.745273:0:14223:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009faa73d4/0xf62499ab9278c2d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da26 expref: 281 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.745285:0:14223:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0000_UUID:14223:1871938510667904:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.745345:0:14223:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.745877:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.745886:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510667776:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.745938:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.746015:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510667776 00000100:00100000:0.0:1785220581.746020:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510667776 00000100:00100000:0.0:1785220581.746061:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284343 00000100:00100000:0.0:1785220581.746069:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97adf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1200:3654:x1871938510667776:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.746146:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29271 count 1 00010000:00010000:0.0:1785220581.746162:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.746181:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000083977864/0xf62499ab9278c378 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da88 expref: 281 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.746189:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7257:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.746193:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7257:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.746197:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.746231:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000083977864/0xf62499ab9278c378 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da88 expref: 281 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.746246:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000083977864/0xf62499ab9278c378 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da88 expref: 281 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.746255:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510668032:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.746298:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.746915:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510668160:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.746949:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.747012:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510668160 00000100:00100000:0.0:1785220581.747017:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510668160 00000100:00100000:0.0:1785220581.747057:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284344 00000100:00100000:0.0:1785220581.747065:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97adf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1201:3652:x1871938510668160:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.747079:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29272 count 1 00010000:00010000:0.0:1785220581.747127:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.747143:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a68e21f9/0xf62499ab9278c46d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7daf1 expref: 281 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.747151:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7258:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.747155:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7258:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.747158:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.747180:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a68e21f9/0xf62499ab9278c46d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7daf1 expref: 281 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.747197:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a68e21f9/0xf62499ab9278c46d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7daf1 expref: 281 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.747219:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510668288:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.747446:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26a80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.747741:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510666752:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.747772:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.747824:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0001_UUID:14259:1871938510666880:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.747838:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.747881:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510667008:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.747893:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.747936:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.747954:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940608 00000100:00100000:1.0:1785220581.747958:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940608 00010000:00010000:1.0:1785220581.747983:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002d75790c/0xf62499ab92789271 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2e1 expref: 323 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748009:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100262 00000100:00100000:1.0:1785220581.748016:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+323:2407:x1871938519940608:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.748026:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.748031:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002d75790c/0xf62499ab92789271 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2e1 expref: 323 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748042:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002d75790c/0xf62499ab92789271 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2e1 expref: 323 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748065:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7222:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.748070:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7222:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.748076:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748107:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748125:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.748159:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+323:2407:x1871938519940608:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 142us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.748168:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100262 00000100:00100000:1.0:1785220581.748285:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748321:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748337:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748353:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748373:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748389:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748480:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748496:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748510:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748527:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.748546:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940864 00000100:00100000:1.0:1785220581.748550:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940864 00010000:00010000:1.0:1785220581.748568:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007c06d4ca/0xf62499ab9278c172 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d99a expref: 282 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748589:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519941120 00000100:00100000:1.0:1785220581.748593:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519941120 00010000:00010000:1.0:1785220581.748603:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d82dbf51/0xf62499ab927893c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c389 expref: 323 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748617:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519941376 00000100:00100000:1.0:1785220581.748619:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519941376 00010000:00010000:1.0:1785220581.748629:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005a7f0339/0xf62499ab927894b6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3dd expref: 324 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748644:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940736 00000100:00100000:1.0:1785220581.748647:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940736 00010000:00010000:1.0:1785220581.748659:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004786e31a/0xf62499ab927891d7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c278 expref: 325 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748680:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519940992 00000100:00100000:1.0:1785220581.748682:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519940992 00010000:00010000:1.0:1785220581.748695:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000278d32e8/0xf62499ab927890b8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c20f expref: 326 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748710:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519941248 00000100:00100000:1.0:1785220581.748713:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519941248 00010000:00010000:1.0:1785220581.748723:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003791778a/0xf62499ab92789319 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c327 expref: 327 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748734:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519941504 00000100:00100000:1.0:1785220581.748736:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519941504 00010000:00010000:1.0:1785220581.748747:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b5ed905a/0xf62499ab9278be5b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d819 expref: 283 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748768:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519941760 00000100:00100000:1.0:1785220581.748772:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519941760 00010000:00010000:1.0:1785220581.748784:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000094899ec8/0xf62499ab9278901e lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c191 expref: 328 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748795:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942016 00000100:00100000:1.0:1785220581.748797:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942016 00010000:00010000:1.0:1785220581.748808:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b300e288/0xf62499ab9278bd2e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7b7 expref: 284 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748817:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519941632 00000100:00100000:1.0:1785220581.748820:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519941632 00010000:00010000:1.0:1785220581.748831:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000bc62747/0xf62499ab9278c061 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8f2 expref: 285 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.748845:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100263 00000100:00100000:1.0:1785220581.748852:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+285:2407:x1871938519940864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.748861:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.748865:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007c06d4ca/0xf62499ab9278c172 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d99a expref: 285 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748875:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007c06d4ca/0xf62499ab9278c172 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d99a expref: 285 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748890:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7255:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.748894:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7255:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.748898:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748906:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748920:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.748945:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+285:2407:x1871938519940864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (660us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.748951:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100263 00000100:00100000:1.0:1785220581.748966:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100264 00000100:00100000:1.0:1785220581.748973:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+328:2407:x1871938519941120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.748980:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.748985:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d82dbf51/0xf62499ab927893c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c389 expref: 328 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.748995:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d82dbf51/0xf62499ab927893c8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c389 expref: 328 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749007:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d82dbf51/0xf62499ab927893c8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c389 expref: 328 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749020:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7224:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.749023:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7224:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.749027:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749035:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749052:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.749067:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+327:2407:x1871938519941120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (746us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.749073:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100264 00000100:00100000:1.0:1785220581.749106:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100265 00000100:00100000:1.0:1785220581.749113:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+326:2407:x1871938519941376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.749123:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.749128:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005a7f0339/0xf62499ab927894b6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3dd expref: 326 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749137:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005a7f0339/0xf62499ab927894b6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3dd expref: 326 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749150:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7225:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.749154:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7225:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.749157:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749164:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.749176:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.749193:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+326:2407:x1871938519941376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (855us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.749199:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100265 00010000:00010000:0.0:1785220581.749344:0:14247:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007c06d4ca/0xf62499ab9278c172 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d99a expref: 284 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.749357:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510666624:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.749374:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749383:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749390:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749399:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749407:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749428:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.749575:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1202:3654:x1871938510665600:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13696us (15151us total) trans 4295070573 rc 0/0 00000100:00100000:0.0:1785220581.749583:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284336 00000100:00100000:0.0:1785220581.749607:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100266 00000100:00100000:0.0:1785220581.749614:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+325:2407:x1871938519940736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.749622:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.749627:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004786e31a/0xf62499ab927891d7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c278 expref: 325 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749637:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004786e31a/0xf62499ab927891d7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c278 expref: 325 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749651:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7221:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.749654:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7221:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.749658:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749666:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749677:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.749693:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+325:2407:x1871938519940736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 80us (1339us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.749703:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100266 00000100:00100000:0.0:1785220581.749716:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100267 00000100:00100000:0.0:1785220581.749722:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce79050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+324:2407:x1871938519940992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.749728:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.749733:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000278d32e8/0xf62499ab927890b8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c20f expref: 324 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749742:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000278d32e8/0xf62499ab927890b8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c20f expref: 324 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749754:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000278d32e8/0xf62499ab927890b8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c20f expref: 324 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749779:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7220:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.749781:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7220:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.749786:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749793:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749809:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.749823:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce79050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+323:2407:x1871938519940992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (1450us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.749828:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100267 00000100:00100000:0.0:1785220581.749858:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100268 00000100:00100000:0.0:1785220581.749864:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+322:2407:x1871938519941248:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.749871:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.749876:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003791778a/0xf62499ab92789319 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c327 expref: 322 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749887:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003791778a/0xf62499ab92789319 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c327 expref: 322 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749899:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003791778a/0xf62499ab92789319 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c327 expref: 322 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749908:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7223:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.749911:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7223:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.749915:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749921:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.749938:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.749956:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+321:2407:x1871938519941248:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (1565us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.749962:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100268 00000100:00100000:0.0:1785220581.749975:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100269 00000100:00100000:0.0:1785220581.749982:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+283:2404:x1871938519941504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.749989:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.749994:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b5ed905a/0xf62499ab9278be5b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d819 expref: 283 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750003:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b5ed905a/0xf62499ab9278be5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d819 expref: 283 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750014:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7252:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.750017:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7252:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.750020:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750028:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750038:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.750053:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+283:2404:x1871938519941504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 72us (1574us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.750060:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100269 00000100:00100000:0.0:1785220581.750073:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100270 00000100:00100000:0.0:1785220581.750080:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+320:2404:x1871938519941760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.750106:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.750112:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000094899ec8/0xf62499ab9278901e lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c191 expref: 320 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750126:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000094899ec8/0xf62499ab9278901e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c191 expref: 320 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750138:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.750141:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x721f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.750145:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750152:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.750163:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.750181:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+320:2404:x1871938519941760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (1684us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.750187:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100270 00000100:00100000:0.0:1785220581.750202:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100271 00010000:00010000:2.0:1785220581.750359:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5ed905a/0xf62499ab9278be5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d819 expref: 282 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.750370:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510664960:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.750388:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750396:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750403:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750411:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750418:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750434:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.750543:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1202:3654:x1871938510664576:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23035us (23239us total) trans 4295070574 rc 0/0 00000100:00100000:2.0:1785220581.750550:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284330 00000100:00100000:2.0:1785220581.750583:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510665088:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.750596:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750622:0:14188:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094899ec8/0xf62499ab9278901e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c191 expref: 319 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.750631:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510665344:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.750641:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750654:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750661:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750668:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750675:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750689:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.750794:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90141c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1135:3653:x1871938510665216:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22805us (22948us total) trans 4295069985 rc 0/0 00000100:00100000:2.0:1785220581.750801:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284332 00010000:00010000:2.0:1785220581.750831:0:14165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004786e31a/0xf62499ab927891d7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c278 expref: 318 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.750843:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510666240:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.750855:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750863:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750870:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750878:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750886:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.750900:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:2.0:1785220581.750998:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1135:3653:x1871938510665856:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17377us (17589us total) trans 4295069986 rc 0/0 00000100:00100000:2.0:1785220581.751005:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284334 00010000:00010000:2.0:1785220581.751023:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220581.751030:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751037:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.751041:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751049:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751062:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751069:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c8c95977/0xf62499ab927a4db6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751106:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.751110:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751116:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.751119:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751127:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751138:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751145:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a4d8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751158:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.751161:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751167:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.751171:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751178:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751189:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.751196:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a4d9a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x721f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.751320:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100272 00000100:00100000:1.0:1785220581.751332:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+281:2404:x1871938519941632:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.751343:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.751350:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000bc62747/0xf62499ab9278c061 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8f2 expref: 281 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.751366:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000bc62747/0xf62499ab9278c061 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8f2 expref: 281 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.751380:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bc62747/0xf62499ab9278c061 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8f2 expref: 281 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.751437:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+280:2404:x1871938519942016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.751447:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.751452:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b300e288/0xf62499ab9278bd2e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7b7 expref: 280 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.751462:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b300e288/0xf62499ab9278bd2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7b7 expref: 280 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.751478:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7251:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.751481:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7251:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.751486:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.751494:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.751509:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.751527:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+280:2404:x1871938519942016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1320us (3015us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.751533:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100271 00010000:00010000:1.0:1785220581.751607:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7254:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.751611:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7254:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.751615:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.751623:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.751644:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.751660:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+279:2404:x1871938519941632:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 332us (3131us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.751666:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100272 00000004:00080000:0.0:1785220581.751808:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94913100/2, rc 0, transno 4295069986 00000100:00100000:0.0:1785220581.751822:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94913100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510665856:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.751833:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7c380/2, rc 0, transno 4295069985 00000100:00100000:0.0:1785220581.751837:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510665216:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.752115:0:14200:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d75790c/0xf62499ab92789271 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c2e1 expref: 317 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.752130:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510666112:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.752147:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752157:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752164:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752220:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752228:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752238:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.752369:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94912300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1135:3653:x1871938510665984:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18573us (18865us total) trans 4295069987 rc 0/0 00000100:00100000:3.0:1785220581.752377:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284335 00010000:00010000:3.0:1785220581.752411:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a7f0339/0xf62499ab927894b6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c3dd expref: 316 pid: 14145 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.752423:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510667264:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.752436:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752444:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752450:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752458:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752465:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.752473:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.752547:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1135:3653:x1871938510667136:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15063us (15156us total) trans 4295069988 rc 0/0 00000100:00100000:3.0:1785220581.752554:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284340 00000004:00080000:0.0:1785220581.752709:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94913480/2, rc 0, transno 4295069988 00000100:00100000:0.0:1785220581.752714:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94913480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510667136:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.752724:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94912d80/2, rc 0, transno 4295069987 00000100:00100000:0.0:1785220581.752727:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94912d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510665984:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.752903:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.752914:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.752922:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.752930:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.752939:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:1.0:1785220581.753029:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1135:3655:x1871938510666496:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16555us (17613us total) trans 4295069989 rc 0/0 00000100:00100000:1.0:1785220581.753034:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284338 00010000:00010000:1.0:1785220581.753051:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753056:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753062:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753066:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753070:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000004:00080000:3.0:1785220581.753472:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94910700/2, rc 0, transno 4295069989 00000100:00100000:3.0:1785220581.753478:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94910700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510666496:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.753585:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1135:3654:x1871938510665728:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17545us (19114us total) trans 4295069990 rc 0/0 00000100:00100000:1.0:1785220581.753592:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284337 00010000:00010000:1.0:1785220581.753612:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753627:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753635:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753642:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753651:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:1.0:1785220581.753736:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94912680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1135:3655:x1871938510666368:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17104us (18273us total) trans 4295069991 rc 0/0 00000100:00100000:1.0:1785220581.753742:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284339 00010000:00010000:1.0:1785220581.753772:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:1.0:1785220581.753779:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753786:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.753790:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753800:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753819:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753827:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a4da8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7221:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753845:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.753848:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753854:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.753857:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753864:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753873:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753881:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a4daf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7222:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753894:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.753897:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753904:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.753907:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753915:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753925:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753933:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a4dd2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7225:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753945:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.753948:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753954:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.753957:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753966:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753976:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753984:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4dc4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7224:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.753997:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.754000:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754007:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.754010:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754017:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754027:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754034:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4dbd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7220:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754046:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.754049:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754056:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.754059:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754071:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754081:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754116:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4dcb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7223:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754129:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.754131:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.754133:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220581.754195:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94913b80/2, rc 0, transno 4295069991 00000100:00100000:3.0:1785220581.754201:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94913b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510666368:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.754347:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.754350:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.754352:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.754364:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754373:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754382:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754389:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754415:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.754568:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.754574:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754581:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.754585:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754593:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754604:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754616:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a4d93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.754631:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.754633:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.754635:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.754767:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1202:3653:x1871938510664832:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27104us (27330us total) trans 4295070575 rc 0/0 00000100:00100000:2.0:1785220581.754774:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284331 00010000:00010000:2.0:1785220581.754800:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.754805:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7f800/2, rc 0, transno 4295070575 00010000:00010000:2.0:1785220581.754807:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.754824:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510664832:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.754828:0:14267:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b300e288/0xf62499ab9278bd2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7b7 expref: 278 pid: 13869 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.754839:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510664448:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.754856:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754865:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754873:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754881:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.754888:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.755022:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510668416:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.755043:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.755061:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.755174:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510668416 00000100:00100000:3.0:1785220581.755181:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510668416 00000100:00100000:3.0:1785220581.755229:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284345 00000100:00100000:3.0:1785220581.755239:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1203:3653:x1871938510668416:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.755263:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29273 count 1 00010000:00010000:3.0:1785220581.755282:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.755297:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000caa54359/0xf62499ab9278c53f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db37 expref: 277 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.755306:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7259:0x0].0x0 since=24s v=(0 0) 00010000:00010000:3.0:1785220581.755310:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7259:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.755313:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.755337:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000caa54359/0xf62499ab9278c53f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db37 expref: 277 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.755348:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000caa54359/0xf62499ab9278c53f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db37 expref: 277 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.755361:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510668544:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.755405:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.755410:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.755418:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.755422:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.755429:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.755441:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.755445:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a4d85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.755455:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.755456:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.755458:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.755666:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1203:3653:x1871938510664320:12345-0@lo:6:osp-syn-0-0.0 Request processed in 28712us (28975us total) trans 4295070576 rc 0/0 00000100:00100000:2.0:1785220581.755674:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284329 00000004:00080000:0.0:1785220581.755688:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7fb80/2, rc 0, transno 4295070576 00000100:00100000:2.0:1785220581.755707:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510665472:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.755707:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510664320:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.755721:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.755773:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.755805:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519941888 00000100:00100000:0.0:1785220581.755809:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519941888 00000100:00100000:2.0:1785220581.755817:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510667520:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.755829:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.755859:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.755918:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.755935:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.755957:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000026e3a62d/0xf62499ab9278bf42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d890 expref: 278 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.755975:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.755987:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942144 00000100:00100000:0.0:1785220581.755991:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942144 00010000:00010000:0.0:1785220581.756006:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005cd1b3be/0xf62499ab927895a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c462 expref: 316 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.756019:0:14223:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0000_UUID:14223:1871938510667904:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.756023:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942272 00000100:00100000:0.0:1785220581.756025:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942272 00010000:00010000:1.0:1785220581.756032:0:14223:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756036:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009faa73d4/0xf62499ab9278c2d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da26 expref: 279 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.756050:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942400 00000100:00100000:0.0:1785220581.756053:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942400 00010000:00010000:0.0:1785220581.756064:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000083977864/0xf62499ab9278c378 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da88 expref: 280 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.756073:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.756079:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942528 00000100:00100000:1.0:1785220581.756081:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510668672:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.756082:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942528 00000100:00100000:1.0:1785220581.756127:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.756287:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a68e21f9/0xf62499ab9278c46d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7daf1 expref: 281 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.756315:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100273 00000100:00100000:0.0:1785220581.756324:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97aded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+281:2404:x1871938519941888:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.756334:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.756336:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510668672 00000100:00100000:2.0:1785220581.756339:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510668672 00010000:00010000:0.0:1785220581.756339:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000026e3a62d/0xf62499ab9278bf42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d890 expref: 281 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756350:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000026e3a62d/0xf62499ab9278bf42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d890 expref: 281 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.756361:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284346 00000100:00100000:2.0:1785220581.756367:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1203:3655:x1871938510668672:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.756368:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000026e3a62d/0xf62499ab9278bf42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d890 expref: 281 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756380:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7253:0x0].0x0 since=0s v=(1 0) 00002000:00080000:2.0:1785220581.756381:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29274 count 1 00010000:00010000:0.0:1785220581.756383:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7253:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.756388:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.756394:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756397:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.756411:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ce4d4012/0xf62499ab9278c6dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dbd1 expref: 280 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756415:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.756418:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x725a:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.756421:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.756425:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 4/0,1 mode: --/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.756454:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97aded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+280:2404:x1871938519941888:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 131us (681us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.756461:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100273 00000100:00100000:3.0:1785220581.756463:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.756475:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100274 00000100:00100000:3.0:1785220581.756480:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100275 00000100:00100000:0.0:1785220581.756482:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9aec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+316:2407:x1871938519942144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.756487:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97ade300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+279:2407:x1871938519942272:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.756489:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.756493:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.756494:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005cd1b3be/0xf62499ab927895a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c462 expref: 316 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756498:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009faa73d4/0xf62499ab9278c2d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da26 expref: 279 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756505:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005cd1b3be/0xf62499ab927895a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c462 expref: 316 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756508:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009faa73d4/0xf62499ab9278c2d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da26 expref: 279 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756519:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009faa73d4/0xf62499ab9278c2d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da26 expref: 279 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756519:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cd1b3be/0xf62499ab927895a4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c462 expref: 316 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756531:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7256:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.756531:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7226:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.756534:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7256:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.756534:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7226:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.756539:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756540:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756549:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756549:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756564:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.756569:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.756584:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97ade300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+278:2407:x1871938519942272:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (665us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.756584:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9aec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+315:2407:x1871938519942144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (724us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.756591:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100275 00000100:00100000:0.0:1785220581.756594:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100274 00000100:00100000:3.0:1785220581.756608:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100276 00000100:00100000:3.0:1785220581.756614:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97addf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+277:2407:x1871938519942400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.756620:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220581.756624:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100277 00010000:00010000:3.0:1785220581.756625:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000083977864/0xf62499ab9278c378 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da88 expref: 277 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.756630:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97adc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+277:2407:x1871938519942528:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.756636:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.756642:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000083977864/0xf62499ab9278c378 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da88 expref: 277 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756642:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a68e21f9/0xf62499ab9278c46d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7daf1 expref: 277 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756651:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a68e21f9/0xf62499ab9278c46d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7daf1 expref: 277 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756655:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7257:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.756658:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7257:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.756662:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756662:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7258:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.756665:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7258:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.756669:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756670:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756677:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.756681:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.756686:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.756698:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97addf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+277:2407:x1871938519942400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (761us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.756698:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97adc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+277:2407:x1871938519942528:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 68us (722us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.756704:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100276 00000100:00100000:0.0:1785220581.756707:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100277 00010000:00010000:0.0:1785220581.756738:0:14208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a68e21f9/0xf62499ab9278c46d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7daf1 expref: 275 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.756747:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ce4d4012/0xf62499ab9278c6dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dbd1 expref: 274 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.756748:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510668288:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.756767:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ce4d4012/0xf62499ab9278c6dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dbd1 expref: 274 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756773:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.756779:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97add180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510668800:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.756787:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756794:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.756802:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.756802:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756810:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.756816:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.756823:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.756824:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.756831:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.756843:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.756865:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.756991:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1135:3653:x1871938510667392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19270us (19325us total) trans 4295069992 rc 0/0 00000100:00100000:2.0:1785220581.756999:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284341 00010000:00010000:2.0:1785220581.757023:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.757028:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97adf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1204:3652:x1871938510668160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9964us (10078us total) trans 4295070577 rc 0/0 00010000:00010000:2.0:1785220581.757032:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.757034:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284344 00010000:00010000:2.0:1785220581.757040:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757048:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757060:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220581.757069:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.757073:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757082:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.757108:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757117:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757130:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757138:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4dee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757155:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.757158:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757165:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.757168:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757176:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757186:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757194:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a4dd9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7226:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757219:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.757222:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757229:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.757233:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757240:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757250:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.757258:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a4da1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.757271:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94912a00/2, rc 0, transno 4295069992 00010000:00010000:2.0:1785220581.757277:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.757278:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94912a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510667392:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.757279:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.757281:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.757309:0:14187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083977864/0xf62499ab9278c378 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da88 expref: 274 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.757319:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510668032:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.757333:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.757348:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.757354:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b09f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1204:3654:x1871938510664704:12345-0@lo:6:osp-syn-0-0.0 Request processed in 29203us (29441us total) trans 4295070578 rc 0/0 00010000:00010000:0.0:1785220581.757355:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.757361:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284333 00010000:00010000:0.0:1785220581.757364:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.757371:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.757386:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.757459:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97adf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1204:3654:x1871938510667776:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11391us (11521us total) trans 4295070579 rc 0/0 00000100:00100000:0.0:1785220581.757465:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284343 00010000:00010000:3.0:1785220581.757670:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.757677:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.757686:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.757689:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.757696:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.757712:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.757719:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a4de7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.757734:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.757736:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.757738:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.757909:0:14223:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.757918:0:14223:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.757925:0:14223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.757932:0:14223:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.757954:0:14223:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.758040:0:14223:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+1204:3654:x1871938510667648:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12908us (13246us total) trans 4295070580 rc 0/0 00000100:00100000:1.0:1785220581.758046:0:14223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284342 00010000:00010000:1.0:1785220581.758066:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.758071:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.758082:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.758105:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.758114:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.758126:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.758133:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4de0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.758147:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.758149:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.758150:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.758156:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.758260:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.758326:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b033100/2, rc 0, transno 4295070577 00000100:00100000:2.0:1785220581.758373:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510668160:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.758464:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.758476:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669056:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.758506:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.758538:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510669056 00000100:00100000:2.0:1785220581.758544:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510669056 00000100:00100000:2.0:1785220581.758577:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284347 00000100:00100000:2.0:1785220581.758587:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97adca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1135:3653:x1871938510669056:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.758609:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29224 count 1 00010000:00010000:2.0:1785220581.758628:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.758646:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cb27f4dc/0xf62499ab92789779 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c565 expref: 314 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.758654:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7228:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.758659:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7228:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.758663:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.758686:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cb27f4dc/0xf62499ab92789779 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c565 expref: 314 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.758698:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cb27f4dc/0xf62499ab92789779 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c565 expref: 314 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.758710:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97ade680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510669184:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.758765:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.758803:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669312:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.758826:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.758881:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.758887:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97addc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669696:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.758909:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.758936:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510669312 00000100:00100000:2.0:1785220581.758940:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510669312 00000100:00100000:2.0:1785220581.758963:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510669696 00000100:00100000:2.0:1785220581.758966:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510669696 00000100:00100000:2.0:1785220581.758986:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284348 00000100:00100000:2.0:1785220581.758992:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1137:3653:x1871938510669312:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.759005:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29225 count 1 00010000:00010000:2.0:1785220581.759016:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.759030:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6e17a64/0xf62499ab9278984b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5a4 expref: 314 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.759037:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7229:0x0].0x0 since=27s v=(0 0) 00010000:00010000:2.0:1785220581.759040:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7229:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.759044:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.759064:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a6e17a64/0xf62499ab9278984b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5a4 expref: 314 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.759077:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a6e17a64/0xf62499ab9278984b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5a4 expref: 314 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.759115:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97add880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510669824:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.759145:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.759179:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97aded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669952:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.759204:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.759320:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510668928:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.759474:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510669952 00000100:00100000:1.0:1785220581.759477:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510669952 00000100:00100000:1.0:1785220581.759505:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284349 00000100:00100000:1.0:1785220581.759512:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97add500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1138:3653:x1871938510669696:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.759524:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29228 count 1 00010000:00010000:1.0:1785220581.759537:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.759549:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007586e6ef/0xf62499ab92789acf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6d8 expref: 314 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.759557:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x722c:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.759560:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.759563:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.759581:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007586e6ef/0xf62499ab92789acf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6d8 expref: 314 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.759590:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007586e6ef/0xf62499ab92789acf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6d8 expref: 314 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.759600:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510670208:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.759630:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.759656:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510668928 00000100:00100000:3.0:1785220581.759659:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510668928 00000100:00100000:3.0:1785220581.759683:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284350 00000100:00100000:3.0:1785220581.759689:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1139:3653:x1871938510669952:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.759700:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29229 count 1 00010000:00010000:3.0:1785220581.759712:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759726:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000465df2a1/0xf62499ab92789bf5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c756 expref: 314 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759733:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x722d:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.759736:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.759741:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759794:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000465df2a1/0xf62499ab92789bf5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c756 expref: 314 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759804:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000465df2a1/0xf62499ab92789bf5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c756 expref: 314 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.759815:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510670336:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.759856:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.759874:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284351 00000100:00100000:3.0:1785220581.759882:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1139:3655:x1871938510668928:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.759893:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29223 count 1 00010000:00010000:3.0:1785220581.759905:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759915:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cb46fb0b/0xf62499ab9278966f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c4e0 expref: 314 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759921:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7227:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.759924:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7227:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.759927:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759945:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cb46fb0b/0xf62499ab9278966f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c4e0 expref: 314 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.759954:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cb46fb0b/0xf62499ab9278966f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c4e0 expref: 314 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.759963:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510670464:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.759985:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.759997:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510669568:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.760020:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.760286:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1785220581.760294:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510669440:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.760318:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.760335:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510670080:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.760357:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.760428:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510669568 00000100:00100000:0.0:1785220581.760433:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510669568 00000100:00100000:0.0:1785220581.760468:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510669440 00000100:00100000:0.0:1785220581.760471:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510669440 00000100:00100000:0.0:1785220581.760490:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510670080 00000100:00100000:0.0:1785220581.760492:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510670080 00000100:00100000:0.0:1785220581.760511:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284352 00000100:00100000:0.0:1785220581.760517:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1141:3655:x1871938510669568:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.760531:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29227 count 1 00010000:00010000:0.0:1785220581.760544:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760555:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b224e95d/0xf62499ab92789a3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c699 expref: 314 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760561:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x722b:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.760564:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.760568:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760587:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b224e95d/0xf62499ab92789a3c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c699 expref: 314 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760594:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b224e95d/0xf62499ab92789a3c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c699 expref: 314 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.760604:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510670592:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.760630:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.760647:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284353 00000100:00100000:0.0:1785220581.760654:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1141:3655:x1871938510669440:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.760666:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29226 count 1 00010000:00010000:0.0:1785220581.760676:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760687:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007d2975c8/0xf62499ab92789955 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c629 expref: 314 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760700:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x722a:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.760703:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.760706:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760725:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007d2975c8/0xf62499ab92789955 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c629 expref: 314 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.760734:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007d2975c8/0xf62499ab92789955 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c629 expref: 314 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.760744:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510670720:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.760878:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.761120:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284354 00000100:00100000:3.0:1785220581.761130:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1204:3655:x1871938510670080:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.761145:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29275 count 1 00010000:00010000:3.0:1785220581.761160:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.761173:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb0c330a/0xf62499ab9278c7fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc56 expref: 273 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.761180:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x725b:0x0].0x0 since=24s v=(0 0) 00010000:00010000:3.0:1785220581.761183:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.761187:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.761401:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cb0c330a/0xf62499ab9278c7fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc56 expref: 273 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.761414:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cb0c330a/0xf62499ab9278c7fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc56 expref: 273 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.761428:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0000_UUID:14185:1871938510670848:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.761469:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.762129:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:2.0:1785220581.762312:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba6300/2, rc 0, transno 4295070579 00000100:00100000:2.0:1785220581.762332:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510667776:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.762346:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba4a80/2, rc 0, transno 4295070580 00000100:00100000:2.0:1785220581.762350:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510667648:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.762363:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94911880/2, rc 0, transno 4295069990 00000100:00100000:2.0:1785220581.762373:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510665728:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.762433:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510670976:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.762465:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.762500:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083ce00/2, rc 0, transno 4295070573 00000100:00100000:2.0:1785220581.762505:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510665600:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.762515:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7f100/2, rc 0, transno 4295070578 00000100:00100000:2.0:1785220581.762519:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510664704:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.762527:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7d500/2, rc 0, transno 4295070574 00000100:00100000:2.0:1785220581.762530:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510664576:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.762648:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510670976 00000100:00100000:0.0:1785220581.762655:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510670976 00000100:00100000:0.0:1785220581.762703:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284355 00000100:00100000:0.0:1785220581.762712:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1142:3655:x1871938510670976:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.762730:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29230 count 1 00010000:00010000:0.0:1785220581.762750:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.762778:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000ceb3f8c/0xf62499ab92789cc0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7d4 expref: 314 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.762793:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x722e:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.762797:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.762802:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.762831:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000ceb3f8c/0xf62499ab92789cc0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7d4 expref: 314 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.762842:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000ceb3f8c/0xf62499ab92789cc0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7d4 expref: 314 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.762853:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510671104:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.762899:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26a80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.762967:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510671232:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.762994:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.763012:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510671744:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.763032:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.763045:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220581.763050:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510671360:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.763070:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.763132:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510671232 00000100:00100000:0.0:1785220581.763136:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510671232 00000100:00100000:0.0:1785220581.763159:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510671744 00000100:00100000:0.0:1785220581.763162:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510671744 00000100:00100000:0.0:1785220581.763176:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510671360 00000100:00100000:0.0:1785220581.763178:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510671360 00000100:00100000:0.0:1785220581.763194:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284356 00000100:00100000:0.0:1785220581.763200:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1207:3652:x1871938510671232:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.763366:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284357 00000100:00100000:3.0:1785220581.763376:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1207:3652:x1871938510671744:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.763390:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29280 count 1 00010000:00010000:3.0:1785220581.763404:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.763416:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b7550cf6/0xf62499ab9278cc2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de40 expref: 273 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.763424:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7260:0x0].0x0 since=24s v=(0 0) 00010000:00010000:3.0:1785220581.763427:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7260:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.763431:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.763451:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b7550cf6/0xf62499ab9278cc2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de40 expref: 273 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.763472:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b7550cf6/0xf62499ab9278cc2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de40 expref: 273 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.763483:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510671872:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.763508:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.763539:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510668544:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.763560:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.763588:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97add180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510668800:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.763601:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.763688:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284358 00000100:00100000:2.0:1785220581.763697:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1207:3652:x1871938510671360:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.763711:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29277 count 1 00010000:00010000:2.0:1785220581.763723:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.763734:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c70485ac/0xf62499ab9278c998 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd13 expref: 273 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.763740:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x725d:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.763743:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.763746:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.763778:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c70485ac/0xf62499ab9278c998 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd13 expref: 273 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.763788:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c70485ac/0xf62499ab9278c998 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd13 expref: 273 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.763798:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510672000:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.763830:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.763861:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510671616:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.763888:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.763923:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.763929:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510671488:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.763950:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.763972:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510671616 00000100:00100000:2.0:1785220581.763976:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510671616 00000100:00100000:2.0:1785220581.764000:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510671488 00000100:00100000:2.0:1785220581.764003:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510671488 00000100:00100000:2.0:1785220581.764017:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284359 00000100:00100000:2.0:1785220581.764022:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1209:3655:x1871938510671616:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.764035:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29279 count 1 00010000:00010000:2.0:1785220581.764046:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.764057:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d71e6cba/0xf62499ab9278cb20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddc2 expref: 273 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.764064:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x725f:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.764067:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.764071:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.764114:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d71e6cba/0xf62499ab9278cb20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddc2 expref: 273 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.764124:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d71e6cba/0xf62499ab9278cb20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddc2 expref: 273 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.764135:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0000_UUID:14172:1871938510672128:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.764162:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.764178:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284360 00000100:00100000:2.0:1785220581.764186:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1209:3655:x1871938510671488:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.764199:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29278 count 1 00000100:00100000:1.0:1785220581.764529:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.764566:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942656 00000100:00100000:1.0:1785220581.764570:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942656 00010000:00010000:1.0:1785220581.764594:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ce4d4012/0xf62499ab9278c6dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dbd1 expref: 274 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.764618:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100278 00000100:00100000:1.0:1785220581.764625:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+274:2407:x1871938519942656:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.764635:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.764640:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ce4d4012/0xf62499ab9278c6dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dbd1 expref: 274 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.764656:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ce4d4012/0xf62499ab9278c6dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dbd1 expref: 274 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.764675:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce4d4012/0xf62499ab9278c6dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dbd1 expref: 274 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.764689:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.764692:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.764697:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.764705:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220581.764936:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29276 count 1 00010000:00010000:0.0:1785220581.764951:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.764964:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8064f4e/0xf62499ab9278c8e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dcb8 expref: 273 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.764971:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x725c:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.764973:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.764977:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.764999:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b8064f4e/0xf62499ab9278c8e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dcb8 expref: 273 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.765009:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b8064f4e/0xf62499ab9278c8e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dcb8 expref: 273 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.765020:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510672256:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.765047:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.765283:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.765298:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ed3b202/0xf62499ab9278ca24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd7c expref: 273 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.765306:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x725e:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.765309:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.765313:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.765335:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ed3b202/0xf62499ab9278ca24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd7c expref: 273 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.765344:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ed3b202/0xf62499ab9278ca24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd7c expref: 273 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.765355:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0000_UUID:14258:1871938510672384:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.765403:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.765543:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.765570:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+273:2407:x1871938519942656:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 945us (3442us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.765580:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100278 00010000:00010000:3.0:1785220581.765841:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.765851:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.765861:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.766401:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.766528:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.766717:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1210:3655:x1871938510668672:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10348us (10587us total) trans 4295070581 rc 0/0 00000100:00100000:2.0:1785220581.766729:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284346 00000004:00080000:0.0:1785220581.766836:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b49180/2, rc 0, transno 4295070581 00000100:00100000:0.0:1785220581.766855:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510668672:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.767045:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510672512:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.767079:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.767188:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510672512 00000100:00100000:1.0:1785220581.767194:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510672512 00000100:00100000:1.0:1785220581.767250:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284361 00000100:00100000:1.0:1785220581.767260:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1210:3653:x1871938510672512:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.767281:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29281 count 1 00010000:00010000:1.0:1785220581.767300:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.767322:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000037c8043d/0xf62499ab9278cd49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7deb7 expref: 272 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.767329:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7261:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.767333:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7261:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.767338:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.767363:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000037c8043d/0xf62499ab9278cd49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7deb7 expref: 272 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.767375:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000037c8043d/0xf62499ab9278cd49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7deb7 expref: 272 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.767387:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510672640:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.767435:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.768073:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.768081:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.768116:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.768120:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.768130:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.768149:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 1/0,0 mode: --/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.768157:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4dfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.768179:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.768181:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.768183:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.770902:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.770963:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942784 00000100:00100000:1.0:1785220581.770973:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942784 00000100:00100000:0.0:1785220581.770998:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.771014:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000caa54359/0xf62499ab9278c53f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db37 expref: 273 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.771020:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.771038:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.771049:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519942912 00000100:00100000:1.0:1785220581.771052:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519942912 00000100:00100000:0.0:1785220581.771055:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943168 00000100:00100000:0.0:1785220581.771057:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943168 00010000:00010000:1.0:1785220581.771067:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a6e17a64/0xf62499ab9278984b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5a4 expref: 316 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.771072:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cb46fb0b/0xf62499ab9278966f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c4e0 expref: 316 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.771173:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510670336:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.771202:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.771208:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97add880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510669824:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.771228:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510670464:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.771228:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.771238:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.771405:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943040 00000100:00100000:0.0:1785220581.771409:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943040 00010000:00010000:0.0:1785220581.771425:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000465df2a1/0xf62499ab92789bf5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c756 expref: 317 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.771460:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100279 00000100:00100000:1.0:1785220581.771470:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99064a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+273:2407:x1871938519942784:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.771485:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.771491:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000caa54359/0xf62499ab9278c53f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db37 expref: 273 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771503:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000caa54359/0xf62499ab9278c53f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db37 expref: 273 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771526:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000caa54359/0xf62499ab9278c53f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db37 expref: 273 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771541:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7259:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.771546:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7259:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.771551:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771561:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771646:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.771696:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99064a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2407:x1871938519942784:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 227us (803us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.771704:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100279 00000100:00100000:1.0:1785220581.771724:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100280 00000100:00100000:1.0:1785220581.771731:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90700450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+317:2406:x1871938519943168:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.771740:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.771745:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cb46fb0b/0xf62499ab9278966f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c4e0 expref: 317 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771755:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cb46fb0b/0xf62499ab9278966f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c4e0 expref: 317 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771780:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cb46fb0b/0xf62499ab9278966f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c4e0 expref: 317 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771791:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7227:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.771794:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7227:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.771798:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771805:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771823:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.771841:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90700450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+316:2406:x1871938519943168:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (819us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.771847:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100280 00000100:00100000:1.0:1785220581.771861:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100281 00000100:00100000:1.0:1785220581.771867:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99067b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+315:2406:x1871938519942912:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.771876:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.771880:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a6e17a64/0xf62499ab9278984b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5a4 expref: 315 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771891:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a6e17a64/0xf62499ab9278984b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5a4 expref: 315 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771903:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6e17a64/0xf62499ab9278984b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c5a4 expref: 315 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771912:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7229:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.771915:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7229:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.771919:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771927:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.771943:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.771961:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99067b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+314:2406:x1871938519942912:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (963us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.771967:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100281 00000100:00100000:1.0:1785220581.772005:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100282 00000100:00100000:1.0:1785220581.772012:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+313:2406:x1871938519943040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.772019:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.772025:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000465df2a1/0xf62499ab92789bf5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c756 expref: 313 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772034:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000465df2a1/0xf62499ab92789bf5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c756 expref: 313 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772048:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000465df2a1/0xf62499ab92789bf5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c756 expref: 313 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772057:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.772060:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.772064:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772072:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772223:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.772246:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+312:2406:x1871938519943040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 234us (1206us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.772253:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100282 00010000:00010000:2.0:1785220581.772311:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.772321:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.772330:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.772338:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.772353:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.772412:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510670592:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.772428:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.772481:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510670720:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.772497:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.772517:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1143:3653:x1871938510669312:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13524us (13689us total) trans 4295069993 rc 0/0 00000100:00100000:2.0:1785220581.772525:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284348 00000100:00100000:2.0:1785220581.772554:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97ade680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510669184:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.772567:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.772597:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adf480/2, rc 0, transno 4295069993 00000100:00100000:2.0:1785220581.772608:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669312:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.772681:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.772695:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510672768:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.772729:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.772849:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772857:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772870:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.772876:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.773079:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.773191:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943296 00000100:00100000:0.0:1785220581.773198:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943296 00010000:00010000:0.0:1785220581.773224:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cb27f4dc/0xf62499ab92789779 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c565 expref: 312 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.773253:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100283 00000100:00100000:0.0:1785220581.773261:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+312:2407:x1871938519943296:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.773272:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.773279:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cb27f4dc/0xf62499ab92789779 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c565 expref: 312 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773288:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cb27f4dc/0xf62499ab92789779 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c565 expref: 312 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773307:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cb27f4dc/0xf62499ab92789779 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c565 expref: 312 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773322:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7228:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.773326:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7228:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.773331:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773341:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773360:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.773375:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.773394:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+311:2407:x1871938519943296:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.773400:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100283 00000100:00100000:0.0:1785220581.773438:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.773456:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943552 00000100:00100000:0.0:1785220581.773459:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943552 00010000:00010000:0.0:1785220581.773474:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007d2975c8/0xf62499ab92789955 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c629 expref: 311 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.773492:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1143:3655:x1871938510668928:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13611us (13862us total) trans 4295069994 rc 0/0 00000100:00100000:0.0:1785220581.773494:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100284 00000100:00100000:1.0:1785220581.773500:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284351 00000100:00100000:0.0:1785220581.773500:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+311:2407:x1871938519943552:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.773507:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.773511:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007d2975c8/0xf62499ab92789955 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c629 expref: 311 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.773516:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510672768 00000100:00100000:1.0:1785220581.773519:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510672768 00010000:00010000:0.0:1785220581.773521:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007d2975c8/0xf62499ab92789955 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c629 expref: 311 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773535:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007d2975c8/0xf62499ab92789955 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c629 expref: 311 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.773539:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284362 00000100:00100000:1.0:1785220581.773545:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b0e1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1143:3655:x1871938510672768:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.773545:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.773548:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.773551:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773559:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220581.773564:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29231 count 1 00010000:00010000:0.0:1785220581.773577:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.773583:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.773593:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+310:2407:x1871938519943552:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (155us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.773595:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000012879ec8/0xf62499ab92789dae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c81a expref: 310 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.773600:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100284 00010000:00010000:1.0:1785220581.773602:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x722f:0x0].0x0 since=27s v=(0 0) 00010000:00010000:1.0:1785220581.773605:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.773609:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773616:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773624:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773632:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.773638:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000012879ec8/0xf62499ab92789dae lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c81a expref: 309 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773639:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.773651:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000012879ec8/0xf62499ab92789dae lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c81a expref: 309 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.773668:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510672896:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.773708:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd780 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.773723:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.773731:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.773740:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.773749:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773789:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220581.773796:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773805:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.773810:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.773811:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220581.773818:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773831:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773838:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4e0a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7229:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773854:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.773858:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773866:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.773869:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773877:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773886:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773893:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a4e1f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7227:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773907:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.773909:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773916:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.773919:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773927:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773936:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773943:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a4e2d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773956:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.773959:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773966:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.773968:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773976:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773984:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.773991:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.773993:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.773995:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.774005:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.774007:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1144:3655:x1871938510669440:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13353us (13687us total) trans 4295069995 rc 0/0 00010000:00010000:2.0:1785220581.774014:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.774014:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284353 00010000:00010000:2.0:1785220581.774027:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774029:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.774036:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.774048:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.774074:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.774193:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97adca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1144:3653:x1871938510669056:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15608us (15685us total) trans 4295069996 rc 0/0 00000100:00100000:2.0:1785220581.774200:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284347 00010000:00010000:2.0:1785220581.774219:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.774224:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.774231:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.774234:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.774241:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.774252:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.774258:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a4e03 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7228:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.774272:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.774274:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.774276:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.774303:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adc700/2, rc 0, transno 4295069996 00000100:00100000:2.0:1785220581.774313:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669056:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.774343:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.774380:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943424 00000100:00100000:0.0:1785220581.774385:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943424 00010000:00010000:1.0:1785220581.774392:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1785220581.774397:0:14196:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a4e18 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774402:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b224e95d/0xf62499ab92789a3c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c699 expref: 310 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.774427:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943680 00000100:00100000:0.0:1785220581.774430:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943680 00010000:00010000:0.0:1785220581.774443:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cb0c330a/0xf62499ab9278c7fb lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc56 expref: 272 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.774463:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100285 00000100:00100000:0.0:1785220581.774470:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+310:2407:x1871938519943424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.774478:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.774482:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b224e95d/0xf62499ab92789a3c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c699 expref: 310 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774492:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b224e95d/0xf62499ab92789a3c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c699 expref: 310 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.774494:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1144:3653:x1871938510669952:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14805us (15289us total) trans 4295069997 rc 0/0 00000100:00100000:1.0:1785220581.774500:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284350 00010000:00010000:0.0:1785220581.774507:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b224e95d/0xf62499ab92789a3c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c699 expref: 310 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774516:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.774520:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.774523:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.774528:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100286 00010000:00010000:0.0:1785220581.774531:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.774535:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2407:x1871938519943680:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.774543:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.774547:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cb0c330a/0xf62499ab9278c7fb lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc56 expref: 272 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774553:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.774558:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cb0c330a/0xf62499ab9278c7fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc56 expref: 272 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774570:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725b:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.774571:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+309:2407:x1871938519943424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (497us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.774573:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725b:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.774577:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100285 00010000:00010000:1.0:1785220581.774578:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774586:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774594:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774597:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.774602:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774610:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.774612:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2407:x1871938519943680:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.774618:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100286 00010000:00010000:0.0:1785220581.774618:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.774630:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.774636:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.774640:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774647:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.774650:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774658:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774669:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774677:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a4e26 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.774690:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.774691:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.774693:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.774695:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1144:3655:x1871938510669568:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14178us (14674us total) trans 4295069998 rc 0/0 00000100:00100000:0.0:1785220581.774701:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284352 00010000:00010000:0.0:1785220581.774715:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.775306:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775317:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775324:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775331:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775348:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.775461:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1211:3653:x1871938510668416:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20224us (20400us total) trans 4295070582 rc 0/0 00000100:00100000:3.0:1785220581.775468:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284345 00010000:00010000:3.0:1785220581.775510:0:14185:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb0c330a/0xf62499ab9278c7fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc56 expref: 271 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.775521:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0000_UUID:14185:1871938510670848:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.775540:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775552:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775558:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775565:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.775572:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776036:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.776042:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776050:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.776054:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776067:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776099:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776108:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a4df5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776127:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.776130:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776137:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.776140:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776147:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776156:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.776163:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.776164:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.776167:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.776203:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.776216:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.776244:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97aded80/2, rc 0, transno 4295069997 00000100:00100000:2.0:1785220581.776251:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97aded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669952:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.776264:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba6a00/2, rc 0, transno 4295070582 00000100:00100000:2.0:1785220581.776308:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510668416:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.776367:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220581.776373:0:14185:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a4e34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.776504:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1211:3655:x1871938510670080:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15376us (16145us total) trans 4295070583 rc 0/0 00000100:00100000:3.0:1785220581.776511:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284354 00000100:00100000:2.0:1785220581.776516:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510672000:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.776532:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.776539:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510671872:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.776550:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.776562:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.776586:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.776604:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.776623:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510671104:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.776636:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0000_UUID:14172:1871938510672128:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.776640:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.776647:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.776666:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943936 00000100:00100000:0.0:1785220581.776671:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943936 00010000:00010000:0.0:1785220581.776695:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b7550cf6/0xf62499ab9278cc2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de40 expref: 271 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.776713:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0000_UUID:14258:1871938510672384:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.776718:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519943808 00000100:00100000:0.0:1785220581.776721:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519943808 00010000:00010000:2.0:1785220581.776727:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.776732:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000ceb3f8c/0xf62499ab92789cc0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7d4 expref: 309 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.776748:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944064 00000100:00100000:0.0:1785220581.776751:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944064 00000100:00100000:2.0:1785220581.776752:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.776777:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c70485ac/0xf62499ab9278c998 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd13 expref: 272 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.776814:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220581.776955:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510673024:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.776991:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.777018:0:14161:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510673024 00000100:00100000:3.0:1785220581.777022:0:14161:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510673024 00000100:00100000:3.0:1785220581.777055:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284363 00000100:00100000:3.0:1785220581.777065:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+1144:3654:x1871938510673024:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.777109:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29232 count 1 00010000:00010000:3.0:1785220581.777197:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.777212:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005f00688e/0xf62499ab92789e48 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c898 expref: 309 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.777220:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7230:0x0].0x0 since=27s v=(0 0) 00010000:00010000:3.0:1785220581.777225:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7230:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.777230:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.777257:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005f00688e/0xf62499ab92789e48 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c898 expref: 309 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.777270:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005f00688e/0xf62499ab92789e48 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c898 expref: 309 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.777282:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0001_UUID:14161:1871938510673152:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.777327:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.777422:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944192 00000100:00100000:0.0:1785220581.777426:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944192 00010000:00010000:0.0:1785220581.777442:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d71e6cba/0xf62499ab9278cb20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddc2 expref: 273 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.777474:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100287 00000100:00100000:3.0:1785220581.777482:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+273:2406:x1871938519943936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.777483:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.777492:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.777498:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b7550cf6/0xf62499ab9278cc2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de40 expref: 273 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.777504:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.777509:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b7550cf6/0xf62499ab9278cc2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de40 expref: 273 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.777528:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7550cf6/0xf62499ab9278cc2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de40 expref: 273 pid: 14186 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.777539:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.777540:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7260:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.777544:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7260:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.777550:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.777558:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.777558:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.777574:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.777594:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2406:x1871938519943936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (1032us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.777601:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100287 00000100:00100000:3.0:1785220581.777613:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944576 00000100:00100000:3.0:1785220581.777616:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944576 00010000:00010000:3.0:1785220581.777628:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ed3b202/0xf62499ab9278ca24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd7c expref: 272 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.777646:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944320 00000100:00100000:3.0:1785220581.777649:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944320 00010000:00010000:3.0:1785220581.777661:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007586e6ef/0xf62499ab92789acf lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6d8 expref: 310 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.777677:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944704 00000100:00100000:3.0:1785220581.777679:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944704 00010000:00010000:3.0:1785220581.777689:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000037c8043d/0xf62499ab9278cd49 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7deb7 expref: 273 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.777705:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944448 00000100:00100000:3.0:1785220581.777707:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944448 00010000:00010000:3.0:1785220581.777719:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b8064f4e/0xf62499ab9278c8e2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dcb8 expref: 274 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.777986:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510672640:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.778006:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.780000:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100288 00000100:00100000:2.0:1785220581.780022:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+310:2406:x1871938519943808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.780041:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.780049:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000ceb3f8c/0xf62499ab92789cc0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7d4 expref: 310 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780063:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000ceb3f8c/0xf62499ab92789cc0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7d4 expref: 310 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.780189:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100289 00000100:00100000:3.0:1785220581.780202:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+274:2406:x1871938519944064:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.780212:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.780218:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c70485ac/0xf62499ab9278c998 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd13 expref: 274 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780228:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c70485ac/0xf62499ab9278c998 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd13 expref: 274 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780247:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70485ac/0xf62499ab9278c998 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd13 expref: 274 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780263:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780267:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780272:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780281:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780307:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.780356:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+273:2406:x1871938519944064:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 159us (3751us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.780364:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100289 00000100:00100000:3.0:1785220581.780385:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100290 00000100:00100000:3.0:1785220581.780393:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2406:x1871938519944192:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.780401:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.780407:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d71e6cba/0xf62499ab9278cb20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddc2 expref: 272 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780417:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d71e6cba/0xf62499ab9278cb20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddc2 expref: 272 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780434:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d71e6cba/0xf62499ab9278cb20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddc2 expref: 272 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780445:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780447:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780451:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780457:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780474:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.780491:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+271:2406:x1871938519944192:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (3738us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.780497:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100290 00000100:00100000:3.0:1785220581.780511:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100291 00000100:00100000:3.0:1785220581.780518:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+270:2406:x1871938519944576:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.780526:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.780531:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ed3b202/0xf62499ab9278ca24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd7c expref: 270 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780541:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ed3b202/0xf62499ab9278ca24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd7c expref: 270 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780559:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ed3b202/0xf62499ab9278ca24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd7c expref: 270 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780569:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780571:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780575:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780583:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780600:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.780618:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+269:2406:x1871938519944576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (3135us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.780626:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100291 00000100:00100000:3.0:1785220581.780645:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100292 00000100:00100000:3.0:1785220581.780652:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+310:2406:x1871938519944320:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.780661:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.780665:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ceb3f8c/0xf62499ab92789cc0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c7d4 expref: 310 pid: 14197 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780666:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007586e6ef/0xf62499ab92789acf lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6d8 expref: 310 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780676:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007586e6ef/0xf62499ab92789acf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6d8 expref: 309 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780678:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.780681:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.780686:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780693:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.780694:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780696:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780700:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780707:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780717:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.780717:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.780733:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+309:2406:x1871938519944320:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (3228us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.780734:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+309:2406:x1871938519943808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 723us (4148us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.780740:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100292 00000100:00100000:2.0:1785220581.780746:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100288 00000100:00100000:3.0:1785220581.780752:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100293 00000100:00100000:3.0:1785220581.780770:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+268:2407:x1871938519944704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.780777:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.780783:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000037c8043d/0xf62499ab9278cd49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7deb7 expref: 268 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780791:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000037c8043d/0xf62499ab9278cd49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7deb7 expref: 268 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.780797:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100294 00010000:00010000:3.0:1785220581.780804:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037c8043d/0xf62499ab9278cd49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7deb7 expref: 268 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.780804:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+268:2405:x1871938519944448:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.780811:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.780816:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b8064f4e/0xf62499ab9278c8e2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dcb8 expref: 268 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.780819:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7261:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.780822:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7261:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.780825:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b8064f4e/0xf62499ab9278c8e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dcb8 expref: 267 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780837:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.780840:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x725c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.780844:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780850:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780866:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.780884:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+267:2405:x1871938519944448:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 81us (3324us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.780891:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100294 00010000:00010000:2.0:1785220581.780919:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780929:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780939:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780947:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.780967:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.781008:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781017:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781028:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781034:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781049:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.781245:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1145:3655:x1871938510670976:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18533us (18778us total) trans 4295069999 rc 0/0 00000100:00100000:0.0:1785220581.781253:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284355 00000100:00100000:2.0:1785220581.781303:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1211:3652:x1871938510671360:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17608us (18232us total) trans 4295070584 rc 0/0 00000100:00100000:2.0:1785220581.781310:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284358 00010000:00010000:2.0:1785220581.781329:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781337:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781348:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781355:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781367:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.781441:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1211:3655:x1871938510671616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17418us (17552us total) trans 4295070585 rc 0/0 00000100:00100000:2.0:1785220581.781448:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284359 00010000:00010000:0.0:1785220581.781464:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8064f4e/0xf62499ab9278c8e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dcb8 expref: 266 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781467:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781475:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.781478:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510672256:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.781483:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781489:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781496:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781502:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:0.0:1785220581.781504:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781510:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781517:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781525:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.781534:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1785220581.781565:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1211:3655:x1871938510671488:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17380us (17614us total) trans 4295070586 rc 0/0 00000100:00100000:2.0:1785220581.781575:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284360 00010000:00010000:2.0:1785220581.781593:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00000100:00100000:0.0:1785220581.781596:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1211:3652:x1871938510671232:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18396us (18601us total) trans 4295070587 rc 0/0 00010000:00010000:2.0:1785220581.781599:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.781602:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284356 00010000:00010000:2.0:1785220581.781608:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.781613:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781621:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781635:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781642:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4e49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.781654:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083ea00/2, rc 0, transno 4295070584 00010000:00010000:2.0:1785220581.781659:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00000100:00100000:0.0:1785220581.781659:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510671360:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.781663:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.781667:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083c380/2, rc 0, transno 4295070587 00010000:00010000:2.0:1785220581.781670:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:0.0:1785220581.781670:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510671232:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.781673:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781685:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781693:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781699:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a4e3b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781711:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.781714:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781720:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.781723:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781730:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.781735:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510673280:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.781739:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.781745:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a4e50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.781777:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.781804:0:14159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510673280 00000100:00100000:0.0:1785220581.781809:0:14159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510673280 00010000:00010000:1.0:1785220581.781980:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.781987:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.781988:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.781996:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.781998:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.781999:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782007:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782017:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.782020:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.782024:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a4e5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782039:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1785220581.782043:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+265:2407:x1871938519944704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1284us (4503us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.782044:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.782051:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100293 00010000:00010000:1.0:1785220581.782051:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.782054:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782061:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782070:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.782073:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782078:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a4e57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.782081:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.782104:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.782112:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.782120:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.782277:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1211:3652:x1871938510671744:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18903us (19243us total) trans 4295070588 rc 0/0 00000100:00100000:3.0:1785220581.782284:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284357 00000100:00100000:0.0:1785220581.782366:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284364 00000100:00100000:0.0:1785220581.782375:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1145:3652:x1871938510673280:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.782395:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00002000:00080000:0.0:1785220581.782398:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29233 count 1 00010000:00010000:1.0:1785220581.782399:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782405:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.782409:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.782415:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782416:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782427:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.782430:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f6b55106/0xf62499ab92789f3d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8fa expref: 307 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782435:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a4e42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.782444:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7231:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.782448:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7231:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.782450:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.782452:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.782452:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782454:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.782463:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782471:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782479:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782487:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.782489:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f6b55106/0xf62499ab92789f3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8fa expref: 307 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782499:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.782503:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f6b55106/0xf62499ab92789f3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8fa expref: 307 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.782514:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510673408:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.782555:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.782571:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1211:3653:x1871938510672512:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15313us (15491us total) trans 4295070589 rc 0/0 00000100:00100000:1.0:1785220581.782577:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284361 00010000:00010000:1.0:1785220581.782593:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.782597:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782604:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.782607:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782614:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.782620:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083e680/2, rc 0, transno 4295070588 00010000:00010000:1.0:1785220581.782624:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.782625:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510671744:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.782631:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4e65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782643:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.782645:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.782647:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.782649:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083d880/2, rc 0, transno 4295070586 00000100:00100000:0.0:1785220581.782653:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510671488:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.782661:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083f480/2, rc 0, transno 4295070585 00000100:00100000:0.0:1785220581.782664:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510671616:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.782669:0:14179:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007586e6ef/0xf62499ab92789acf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c6d8 expref: 307 pid: 14247 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.782673:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99918700/2, rc 0, transno 4295069999 00000100:00100000:1.0:1785220581.782679:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510670208:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.782681:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510670976:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.782690:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adea00/2, rc 0, transno 4295069995 00010000:00010000:1.0:1785220581.782693:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.782694:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510669440:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.782700:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.782702:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b49500/2, rc 0, transno 4295070583 00000100:00100000:0.0:1785220581.782705:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510670080:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.782708:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.782713:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adfb80/2, rc 0, transno 4295069994 00000100:00100000:0.0:1785220581.782716:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510668928:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.782721:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.782724:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adce00/2, rc 0, transno 4295069998 00000100:00100000:0.0:1785220581.782727:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510669568:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.782729:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.782740:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.782926:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.782931:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.782938:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.782942:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.782950:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.782961:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.782968:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4e11 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.782981:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.782983:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.782984:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.782988:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510673920:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.782991:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.782997:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.783011:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510673536:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.783017:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.783108:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510673792:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.783143:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.783319:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97add500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1146:3653:x1871938510669696:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23807us (24408us total) trans 4295070000 rc 0/0 00000100:00100000:1.0:1785220581.783325:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284349 00000100:00100000:2.0:1785220581.783413:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.783448:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.783455:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510673664:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.783476:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.783478:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510673920 00000100:00100000:1.0:1785220581.783483:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510673920 00000100:00100000:1.0:1785220581.783506:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510673792 00000100:00100000:1.0:1785220581.783508:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510673792 00000100:00100000:1.0:1785220581.783522:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510673664 00000100:00100000:1.0:1785220581.783524:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510673664 00000100:00100000:1.0:1785220581.783540:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284365 00000100:00100000:1.0:1785220581.783546:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1148:3652:x1871938510673920:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.783562:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29237 count 1 00010000:00010000:1.0:1785220581.783576:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783591:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004bcb20a5/0xf62499ab9278a334 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cadd expref: 306 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783599:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7235:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.783602:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7235:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.783607:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783627:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004bcb20a5/0xf62499ab9278a334 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cadd expref: 306 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783638:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004bcb20a5/0xf62499ab9278a334 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cadd expref: 306 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.783649:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510674048:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.783684:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdb40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.783708:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510672896:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.783725:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.783746:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284366 00000100:00100000:1.0:1785220581.783754:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1148:3655:x1871938510673792:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.783778:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29236 count 1 00010000:00010000:1.0:1785220581.783790:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783801:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e5b17f2c/0xf62499ab9278a231 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca6d expref: 306 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783815:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7234:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.783818:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7234:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.783821:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783839:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e5b17f2c/0xf62499ab9278a231 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca6d expref: 306 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783848:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e5b17f2c/0xf62499ab9278a231 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca6d expref: 306 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.783858:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510674176:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.783880:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.783896:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944960 00000100:00100000:1.0:1785220581.783899:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944960 00010000:00010000:1.0:1785220581.783917:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000012879ec8/0xf62499ab92789dae lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c81a expref: 307 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.783939:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100295 00000100:00100000:1.0:1785220581.783946:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e13450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+307:2407:x1871938519944960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.783955:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.783960:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000012879ec8/0xf62499ab92789dae lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c81a expref: 307 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783975:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000012879ec8/0xf62499ab92789dae lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c81a expref: 307 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.783992:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000012879ec8/0xf62499ab92789dae lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c81a expref: 307 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784002:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.784005:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x722f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.784010:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784018:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784035:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.784055:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e13450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+306:2407:x1871938519944960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (643us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.784062:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100295 00000100:00100000:0.0:1785220581.784119:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.784127:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519944832 00000100:00100000:1.0:1785220581.784130:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519944832 00010000:00010000:1.0:1785220581.784144:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005f00688e/0xf62499ab92789e48 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c898 expref: 306 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.784165:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100296 00000100:00100000:1.0:1785220581.784172:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+306:2407:x1871938519944832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.784180:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.784185:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005f00688e/0xf62499ab92789e48 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c898 expref: 306 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784194:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005f00688e/0xf62499ab92789e48 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c898 expref: 306 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784281:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7230:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.784286:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7230:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.784291:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784299:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784312:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.784331:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+306:2407:x1871938519944832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 160us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.784338:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100296 00010000:00010000:1.0:1785220581.784362:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784371:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784380:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784389:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784407:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.784523:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b0e1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1149:3655:x1871938510672768:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10978us (11794us total) trans 4295070001 rc 0/0 00000100:00100000:1.0:1785220581.784530:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284362 00000100:00100000:1.0:1785220581.784616:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284367 00000100:00100000:1.0:1785220581.784622:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1148:3655:x1871938510673664:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.784635:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29235 count 1 00010000:00010000:1.0:1785220581.784646:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784657:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000038e1c144/0xf62499ab9278a104 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9e1 expref: 305 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784664:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7233:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.784667:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7233:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.784670:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784689:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000038e1c144/0xf62499ab9278a104 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9e1 expref: 305 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784698:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000038e1c144/0xf62499ab9278a104 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9e1 expref: 305 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.784709:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510674304:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.784736:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fc00 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.784935:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.784942:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784950:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.784954:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784963:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784976:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.784982:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a4e6c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x722f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.785001:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.785002:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.785004:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.785010:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.785514:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510674816:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.785555:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.785665:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1785220581.785672:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510674432:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.785697:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.785713:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510674944:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.785731:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.785771:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510674560:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.785793:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.785810:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510675072:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.786425:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510674816 00000100:00100000:0.0:1785220581.786433:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510674816 00000100:00100000:0.0:1785220581.786470:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510674432 00000100:00100000:0.0:1785220581.786473:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510674432 00000100:00100000:0.0:1785220581.786490:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510674944 00000100:00100000:0.0:1785220581.786493:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510674944 00000100:00100000:0.0:1785220581.786510:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510674560 00000100:00100000:0.0:1785220581.786513:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510674560 00000100:00100000:0.0:1785220581.786542:0:14169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284368 00000100:00100000:0.0:1785220581.786550:0:14169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1214:3652:x1871938510674816:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.786572:0:14169:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29285 count 1 00010000:00010000:0.0:1785220581.786594:0:14169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.786614:0:14169:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000045f73b4e/0xf62499ab9278d186 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0af expref: 264 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.786622:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7265:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.786626:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7265:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.786631:0:14169:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.786660:0:14169:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000045f73b4e/0xf62499ab9278d186 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0af expref: 264 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.786672:0:14169:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000045f73b4e/0xf62499ab9278d186 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0af expref: 264 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.786684:0:14169:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0000_UUID:14169:1871938510675328:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.786724:0:14169:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26a80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.787540:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284369 00000100:00100000:0.0:1785220581.787552:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1214:3652:x1871938510674432:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.787571:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29282 count 1 00010000:00010000:0.0:1785220581.787586:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.787602:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f25df189/0xf62499ab9278ce99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df35 expref: 264 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.787610:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7262:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.787614:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7262:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.787617:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.787634:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.787640:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f25df189/0xf62499ab9278ce99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df35 expref: 264 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.787650:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f25df189/0xf62499ab9278ce99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df35 expref: 264 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.787661:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510675456:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.787675:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1785220581.787681:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510674688:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.787691:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.787704:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510675072 00000100:00100000:0.0:1785220581.787708:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510675072 00000100:00100000:1.0:1785220581.787712:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.787730:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510675200:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.787733:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510674688 00000100:00100000:0.0:1785220581.787736:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510674688 00000100:00100000:1.0:1785220581.787750:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.787753:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284370 00000100:00100000:0.0:1785220581.787774:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1216:3652:x1871938510674944:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.787787:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29286 count 1 00010000:00010000:0.0:1785220581.787798:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.787810:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000215d0023/0xf62499ab9278d29e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e118 expref: 264 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.787822:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7266:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.788187:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7266:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.788192:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.788216:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000215d0023/0xf62499ab9278d29e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e118 expref: 264 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.788226:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000215d0023/0xf62499ab9278d29e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e118 expref: 264 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.788237:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0000_UUID:14145:1871938510675584:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.788445:0:14175:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510675200 00000100:00100000:3.0:1785220581.788450:0:14175:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510675200 00000100:00100000:2.0:1785220581.788480:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284371 00000100:00100000:0.0:1785220581.788488:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.788490:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1217:3654:x1871938510674560:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.788504:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284372 00002000:00080000:2.0:1785220581.788505:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29283 count 1 00000100:00100000:0.0:1785220581.788510:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1217:3654:x1871938510675072:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.788518:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220581.788520:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29287 count 1 00010000:00010000:0.0:1785220581.788530:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788531:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007a3a4130/0xf62499ab9278cfaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfcf expref: 264 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788538:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7263:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.788541:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7263:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.788541:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd2633e0/0xf62499ab9278d385 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e188 expref: 264 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788545:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.788548:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7267:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.788551:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7267:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.788555:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788564:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007a3a4130/0xf62499ab9278cfaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfcf expref: 264 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.788570:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cd2633e0/0xf62499ab9278d385 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e188 expref: 264 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788573:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007a3a4130/0xf62499ab9278cfaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfcf expref: 264 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.788579:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cd2633e0/0xf62499ab9278d385 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e188 expref: 264 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.788584:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510675712:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.788587:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0000_UUID:14260:1871938510675840:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.788608:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.788625:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284373 00000100:00100000:2.0:1785220581.788631:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1217:3654:x1871938510674688:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.788640:0:14175:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284374 00002000:00080000:2.0:1785220581.788642:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29284 count 1 00000100:00100000:3.0:1785220581.788646:0:14175:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1217:3654:x1871938510675200:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.788652:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.788654:0:14175:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29288 count 1 00010000:00010000:3.0:1785220581.788665:0:14175:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788665:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005291ffe6/0xf62499ab9278d09f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e03f expref: 264 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788673:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7264:0x0].0x0 since=24s v=(0 0) 00010000:00010000:3.0:1785220581.788676:0:14175:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fdeb20ff/0xf62499ab9278d457 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e206 expref: 264 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788676:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7264:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.788680:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.788684:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7268:0x0].0x0 since=24s v=(0 0) 00010000:00010000:3.0:1785220581.788686:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7268:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.788690:0:14175:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788697:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005291ffe6/0xf62499ab9278d09f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e03f expref: 264 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.788704:0:14175:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fdeb20ff/0xf62499ab9278d457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e206 expref: 264 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.788706:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005291ffe6/0xf62499ab9278d09f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e03f expref: 264 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.788713:0:14175:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fdeb20ff/0xf62499ab9278d457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e206 expref: 264 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.788716:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94910380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510675968:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.788722:0:14175:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0000_UUID:14175:1871938510676096:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.788737:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.788741:0:14175:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.788774:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.788785:0:14161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005f00688e/0xf62499ab92789e48 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c898 expref: 305 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.788799:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0001_UUID:14161:1871938510673152:192.168.201.51@tcp:104: 00000004:00080000:2.0:1785220581.788812:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4b100/2, rc 0, transno 4295070589 00010000:00010000:3.0:1785220581.788818:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.788824:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510672512:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.788837:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97addc00/2, rc 0, transno 4295070000 00000100:00100000:2.0:1785220581.788846:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97addc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510669696:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.788901:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.788908:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94912300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510676224:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.788935:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.788949:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a4d5600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.788965:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510673536 00000100:00100000:0.0:1785220581.788968:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510673536 00000100:00100000:0.0:1785220581.788993:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510676224 00000100:00100000:2.0:1785220581.788996:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94910a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510676352:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.788996:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510676224 00000100:00100000:0.0:1785220581.789012:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284375 00000100:00100000:2.0:1785220581.789016:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.789019:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1149:3653:x1871938510673536:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.789031:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29234 count 1 00010000:00010000:0.0:1785220581.789045:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.789056:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000023fee525/0xf62499ab9278a05c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c98d expref: 304 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.789063:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7232:0x0].0x0 since=27s v=(0 0) 00010000:00010000:0.0:1785220581.789066:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7232:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.789069:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.789185:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.789195:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.789205:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.789212:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.789234:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.789278:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510676352 00000100:00100000:1.0:1785220581.789282:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510676352 00000100:00100000:1.0:1785220581.789306:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284376 00000100:00100000:1.0:1785220581.789313:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1218:3653:x1871938510676224:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.789324:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29289 count 1 00010000:00010000:1.0:1785220581.789336:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.789347:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f2f1e6/0xf62499ab9278d5ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e26f expref: 264 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.789357:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7269:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.789360:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7269:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.789363:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.789381:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a7f2f1e6/0xf62499ab9278d5ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e26f expref: 264 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.789388:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+1151:3654:x1871938510673024:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12325us (12397us total) trans 4295070002 rc 0/0 00010000:00010000:1.0:1785220581.789392:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a7f2f1e6/0xf62499ab9278d5ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e26f expref: 264 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.789396:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284363 00000100:00100000:1.0:1785220581.789402:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0000_UUID:14266:1871938510676608:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.789416:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284377 00000100:00100000:3.0:1785220581.789422:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+1150:3653:x1871938510676352:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.789425:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f840 going to sleep for 11 seconds 00002000:00080000:3.0:1785220581.789434:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29238 count 1 00010000:00010000:3.0:1785220581.789445:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.789450:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04a80/2, rc 0, transno 4295070002 00010000:00010000:3.0:1785220581.789456:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009667deb7/0xf62499ab9278a414 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb15 expref: 304 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.789457:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510673024:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.789463:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7236:0x0].0x0 since=26s v=(0 0) 00010000:00010000:3.0:1785220581.789466:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7236:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.789469:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.789505:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000023fee525/0xf62499ab9278a05c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c98d expref: 304 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.789516:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000023fee525/0xf62499ab9278a05c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c98d expref: 304 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.789525:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.789531:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.789539:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.789543:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.789548:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009667deb7/0xf62499ab9278a414 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb15 expref: 304 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.789552:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.789558:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009667deb7/0xf62499ab9278a414 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb15 expref: 304 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.789567:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0001_UUID:14161:1871938510676736:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.789570:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.789577:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a4e73 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7230:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.789584:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.789596:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.789598:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.789600:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.789641:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510676480:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.789659:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9aedcd80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.789677:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.789689:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0001_UUID:14159:1871938510673408:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.789704:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.789723:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945088 00000100:00100000:0.0:1785220581.789726:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945088 00010000:00010000:0.0:1785220581.789745:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f6b55106/0xf62499ab92789f3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8fa expref: 305 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.789777:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.789780:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100297 00000100:00100000:0.0:1785220581.789787:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+305:2407:x1871938519945088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.789796:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.789802:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f6b55106/0xf62499ab92789f3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8fa expref: 305 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.789818:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f6b55106/0xf62499ab92789f3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8fa expref: 305 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.789850:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.790081:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510674176:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.790240:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.790267:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510674048:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.790278:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.790301:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94912680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510676864:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.790330:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.790430:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510676864 00000100:00100000:2.0:1785220581.790435:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510676864 00000100:00100000:2.0:1785220581.790745:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284378 00000100:00100000:2.0:1785220581.790754:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1151:3654:x1871938510676864:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.790784:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29239 count 1 00010000:00010000:2.0:1785220581.790800:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.790814:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c37b7809/0xf62499ab9278a59c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cbb6 expref: 305 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.790821:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7237:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.790824:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7237:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.790828:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.790855:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c37b7809/0xf62499ab9278a59c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cbb6 expref: 305 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.790865:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c37b7809/0xf62499ab9278a59c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cbb6 expref: 305 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.790876:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510676992:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.790922:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.791017:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f6b55106/0xf62499ab92789f3d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c8fa expref: 305 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791033:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7231:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.791036:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7231:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.791042:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791050:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791072:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.791111:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+304:2407:x1871938519945088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1324us (1433us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.791119:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100297 00000100:00100000:0.0:1785220581.791135:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945216 00000100:00100000:0.0:1785220581.791139:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945216 00010000:00010000:0.0:1785220581.791154:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004bcb20a5/0xf62499ab9278a334 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cadd expref: 304 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.791183:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945344 00000100:00100000:0.0:1785220581.791186:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945344 00010000:00010000:0.0:1785220581.791200:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e5b17f2c/0xf62499ab9278a231 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca6d expref: 305 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.791213:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100298 00000100:00100000:0.0:1785220581.791218:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100299 00000100:00100000:1.0:1785220581.791222:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+305:2407:x1871938519945216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.791223:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+305:2407:x1871938519945344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.791230:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.791231:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.791236:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004bcb20a5/0xf62499ab9278a334 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cadd expref: 305 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791236:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e5b17f2c/0xf62499ab9278a231 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca6d expref: 305 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791248:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004bcb20a5/0xf62499ab9278a334 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cadd expref: 305 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791248:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e5b17f2c/0xf62499ab9278a231 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca6d expref: 305 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791263:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004bcb20a5/0xf62499ab9278a334 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cadd expref: 305 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791264:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e5b17f2c/0xf62499ab9278a231 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ca6d expref: 305 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791275:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7235:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.791275:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7234:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.791279:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7235:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.791279:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7234:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.791284:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791284:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791294:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791294:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791310:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.791335:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+303:2407:x1871938519945344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (1485us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.791344:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100299 00010000:00010000:0.0:1785220581.791368:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791377:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791386:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791394:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.791413:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.791451:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.791471:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+302:2407:x1871938519945216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 251us (1694us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.791477:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100298 00010000:00010000:1.0:1785220581.791497:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791505:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791512:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791518:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791529:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.791553:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb22a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+1152:3652:x1871938510673280:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9179us (9775us total) trans 4295070003 rc 0/0 00000100:00100000:0.0:1785220581.791577:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284364 00000100:00100000:1.0:1785220581.791601:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1152:3655:x1871938510673792:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7848us (8457us total) trans 4295070004 rc 0/0 00000100:00100000:1.0:1785220581.791607:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284366 00010000:00010000:1.0:1785220581.791628:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.791633:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791641:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.791645:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791658:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791671:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791678:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a4e7a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7231:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791693:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.791697:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.791701:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb20380/2, rc 0, transno 4295070004 00010000:00010000:1.0:1785220581.791702:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.791705:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.791711:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510673792:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.791712:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791721:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.791723:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90141c00/2, rc 0, transno 4295070001 00000100:00100000:0.0:1785220581.791727:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510672768:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.791729:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a4e88 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7234:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791745:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.791747:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.791748:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.791768:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791776:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791784:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791791:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791803:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.791872:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93666300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1152:3652:x1871938510673920:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8325us (8854us total) trans 4295070005 rc 0/0 00000100:00100000:1.0:1785220581.791879:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284365 00010000:00010000:1.0:1785220581.791895:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.791900:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791906:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.791909:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791917:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791928:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791935:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075879d7f/0xf62499ab927a4e81 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7235:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.791949:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.791951:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.791952:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.792157:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.792330:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb21180/2, rc 0, transno 4295070005 00000100:00100000:2.0:1785220581.792336:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510673920:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.792346:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb36d80/2, rc 0, transno 4295070003 00000100:00100000:2.0:1785220581.792350:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510673280:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.792427:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510677120:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.792460:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.792490:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.792496:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510677504:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.792501:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510677376:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.792517:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.792537:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.792558:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510677120 00000100:00100000:2.0:1785220581.792563:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510677120 00000100:00100000:3.0:1785220581.792575:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510677248:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.792602:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.794959:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510677504 00000100:00100000:2.0:1785220581.794970:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510677504 00000100:00100000:3.0:1785220581.795083:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510677376 00000100:00100000:3.0:1785220581.795101:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510677376 00000100:00100000:3.0:1785220581.795144:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510677248 00000100:00100000:3.0:1785220581.795148:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510677248 00000100:00100000:2.0:1785220581.795159:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284379 00000100:00100000:2.0:1785220581.795169:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1155:3655:x1871938510677120:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.795199:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29240 count 1 00010000:00010000:2.0:1785220581.795226:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.795249:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000631cb17a/0xf62499ab9278a6d7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc65 expref: 301 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.795257:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7238:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.795262:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7238:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.795266:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.795300:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000631cb17a/0xf62499ab9278a6d7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc65 expref: 301 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.795313:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000631cb17a/0xf62499ab9278a6d7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc65 expref: 301 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.795332:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb66680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510677632:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.795372:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284380 00000100:00100000:3.0:1785220581.795380:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1155:3655:x1871938510677504:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.795393:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29243 count 1 00010000:00010000:3.0:1785220581.795406:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.795410:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.795419:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004c39cec8/0xf62499ab9278a9bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdca expref: 301 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.795425:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x723b:0x0].0x0 since=26s v=(0 0) 00010000:00010000:3.0:1785220581.795428:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723b:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220581.795431:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284381 00010000:00010000:3.0:1785220581.795432:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.795438:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1155:3653:x1871938510677376:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.795450:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004c39cec8/0xf62499ab9278a9bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdca expref: 301 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.795450:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29242 count 1 00010000:00010000:3.0:1785220581.795460:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004c39cec8/0xf62499ab9278a9bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdca expref: 301 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.795462:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.795470:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510677760:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.795473:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ffd56c22/0xf62499ab9278a8f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd61 expref: 301 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.795480:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x723a:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.795483:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.795486:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.795494:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.795503:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ffd56c22/0xf62499ab9278a8f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd61 expref: 301 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.795512:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ffd56c22/0xf62499ab9278a8f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd61 expref: 301 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.795522:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb66a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510677888:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.795544:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.795609:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284382 00000100:00100000:1.0:1785220581.795620:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1155:3652:x1871938510677248:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.795632:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29241 count 1 00010000:00010000:1.0:1785220581.795644:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.795657:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000073a5b953/0xf62499ab9278a7d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ccd5 expref: 301 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.795664:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7239:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.795667:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7239:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.795669:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.795686:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000073a5b953/0xf62499ab9278a7d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ccd5 expref: 301 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.795693:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000073a5b953/0xf62499ab9278a7d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ccd5 expref: 301 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.795699:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510678016:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.795716:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f240 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.797016:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.797051:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510674304:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.797078:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.797121:0:14169:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0000_UUID:14169:1871938510675328:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.797147:0:14169:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.797297:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.797560:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510675456:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.797575:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.797623:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0000_UUID:14145:1871938510675584:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.797636:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.798204:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510675712:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.798226:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.798417:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94910380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510675968:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.798448:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0000_UUID:14260:1871938510675840:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.798466:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.798494:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.798692:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0000_UUID:14266:1871938510676608:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.798744:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.799411:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945600 00000100:00100000:3.0:1785220581.799420:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945600 00010000:00010000:3.0:1785220581.799462:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000045f73b4e/0xf62499ab9278d186 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0af expref: 265 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.799506:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100300 00000100:00100000:3.0:1785220581.799518:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+265:2407:x1871938519945600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.799533:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.799540:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000045f73b4e/0xf62499ab9278d186 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0af expref: 265 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.799552:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000045f73b4e/0xf62499ab9278d186 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0af expref: 265 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.799576:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045f73b4e/0xf62499ab9278d186 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0af expref: 265 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.799594:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7265:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.799599:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7265:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.799605:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.799614:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.799639:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.799675:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945472 00000100:00100000:1.0:1785220581.799680:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945472 00000100:00100000:3.0:1785220581.799687:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+264:2407:x1871938519945600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 171us (2676us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.799695:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100300 00010000:00010000:1.0:1785220581.799702:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000038e1c144/0xf62499ab9278a104 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9e1 expref: 302 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.799731:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100301 00010000:00010000:0.0:1785220581.799800:0:14169:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.799810:0:14169:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.799821:0:14169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.799962:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+302:2407:x1871938519945472:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.799972:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.799978:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000038e1c144/0xf62499ab9278a104 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9e1 expref: 302 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.799988:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000038e1c144/0xf62499ab9278a104 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9e1 expref: 302 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.800003:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038e1c144/0xf62499ab9278a104 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c9e1 expref: 302 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.800015:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7233:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.800019:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7233:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.800024:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.800032:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.800052:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.800073:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+301:2407:x1871938519945472:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 336us (2776us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.800122:0:14169:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.800144:0:14169:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.800311:0:14169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1219:3652:x1871938510674816:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13761us (14754us total) trans 4295070590 rc 0/0 00000100:00100000:0.0:1785220581.800319:0:14169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284368 00010000:00010000:0.0:1785220581.800414:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.800420:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.800428:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.800432:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.800441:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.800454:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.800461:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4e96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.800480:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.800483:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.800485:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.801614:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.801628:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.801775:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.801785:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.801808:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.801908:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.801915:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.801923:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.801926:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.801941:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.801960:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.801967:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a4e8f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7233:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.801986:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.801988:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.801989:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.802187:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1156:3655:x1871938510673664:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17561us (18707us total) trans 4295070006 rc 0/0 00000100:00100000:0.0:1785220581.802197:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284367 00000004:00080000:1.0:1785220581.802238:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb20a80/2, rc 0, transno 4295070006 00000100:00100000:1.0:1785220581.802251:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510673664:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.802330:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220581.802344:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510678144:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.802381:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.802474:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100301 00000100:00100000:3.0:1785220581.802503:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510678144 00000100:00100000:3.0:1785220581.802510:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510678144 00000100:00100000:3.0:1785220581.802549:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284383 00000100:00100000:3.0:1785220581.802556:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1156:3654:x1871938510678144:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.802582:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29244 count 1 00010000:00010000:3.0:1785220581.802601:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.802615:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006b08d1da/0xf62499ab9278aa11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce10 expref: 300 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.802623:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x723c:0x0].0x0 since=26s v=(0 0) 00010000:00010000:3.0:1785220581.802627:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.802631:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.802652:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006b08d1da/0xf62499ab9278aa11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce10 expref: 300 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.802663:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006b08d1da/0xf62499ab9278aa11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce10 expref: 300 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.802674:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510678272:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.802737:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.802780:0:14175:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0000_UUID:14175:1871938510676096:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.802799:0:14175:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220581.802847:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b48e00/2, rc 0, transno 4295070590 00000100:00100000:3.0:1785220581.802865:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510674816:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.802885:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0001_UUID:14161:1871938510676736:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.802897:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.802955:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.802989:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.803006:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.803024:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945728 00000100:00100000:1.0:1785220581.803028:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945728 00010000:00010000:1.0:1785220581.803047:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f25df189/0xf62499ab9278ce99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df35 expref: 264 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.803072:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945984 00000100:00100000:1.0:1785220581.803075:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945984 00010000:00010000:1.0:1785220581.803112:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007a3a4130/0xf62499ab9278cfaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfcf expref: 265 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.803136:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519946240 00000100:00100000:1.0:1785220581.803139:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519946240 00010000:00010000:1.0:1785220581.803151:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fdeb20ff/0xf62499ab9278d457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e206 expref: 266 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.803168:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100302 00000100:00100000:1.0:1785220581.803175:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ad050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+266:2406:x1871938519945728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.803187:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.803192:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f25df189/0xf62499ab9278ce99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df35 expref: 266 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.803202:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100303 00010000:00010000:1.0:1785220581.803204:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f25df189/0xf62499ab9278ce99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df35 expref: 266 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.803210:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+266:2406:x1871938519945984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.803221:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f25df189/0xf62499ab9278ce99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df35 expref: 266 pid: 14149 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803233:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7262:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.803236:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7262:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.803237:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.803242:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.803243:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007a3a4130/0xf62499ab9278cfaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfcf expref: 265 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803252:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.803253:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007a3a4130/0xf62499ab9278cfaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfcf expref: 265 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803271:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.803271:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a3a4130/0xf62499ab9278cfaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfcf expref: 265 pid: 14221 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.803280:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7263:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.803283:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7263:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.803287:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.803294:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ad050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+264:2406:x1871938519945728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 120us (340us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.803295:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.803301:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100302 00010000:00010000:0.0:1785220581.803312:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.803328:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+263:2406:x1871938519945984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 120us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.803334:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100303 00000100:00100000:1.0:1785220581.803344:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100304 00000100:00100000:1.0:1785220581.803350:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+262:2406:x1871938519946240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.803358:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.803362:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fdeb20ff/0xf62499ab9278d457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e206 expref: 262 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803372:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fdeb20ff/0xf62499ab9278d457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e206 expref: 262 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803384:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fdeb20ff/0xf62499ab9278d457 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e206 expref: 262 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803394:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7268:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.803396:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7268:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.803400:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803412:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.803428:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.803440:0:14175:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.803446:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+261:2406:x1871938519946240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (440us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.803449:0:14175:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.803463:0:14175:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.803472:0:14175:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.803487:0:14175:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.803621:0:14175:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1219:3654:x1871938510675200:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14969us (15864us total) trans 4295070591 rc 0/0 00000100:00100000:3.0:1785220581.803627:0:14175:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284374 00000100:00100000:3.0:1785220581.803813:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510678400:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.803851:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.803953:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.804010:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510676992:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.804026:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.804065:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.804504:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb66680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510677632:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.804519:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.804542:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.804558:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.804575:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.804612:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb66a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510677888:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.804622:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.804673:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.804686:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519946496 00000100:00100000:2.0:1785220581.804690:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519946496 00010000:00010000:2.0:1785220581.804710:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f2f1e6/0xf62499ab9278d5ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e26f expref: 262 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.804775:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519945856 00000100:00100000:0.0:1785220581.804779:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519945856 00010000:00010000:0.0:1785220581.804793:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000215d0023/0xf62499ab9278d29e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e118 expref: 263 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.804812:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519946112 00000100:00100000:0.0:1785220581.804814:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519946112 00010000:00010000:0.0:1785220581.804835:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005291ffe6/0xf62499ab9278d09f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e03f expref: 264 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.804850:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519946368 00000100:00100000:0.0:1785220581.804853:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519946368 00010000:00010000:0.0:1785220581.804863:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cd2633e0/0xf62499ab9278d385 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e188 expref: 265 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.804877:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519946624 00000100:00100000:0.0:1785220581.804879:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519946624 00010000:00010000:0.0:1785220581.804889:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009667deb7/0xf62499ab9278a414 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb15 expref: 301 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.804901:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519946880 00000100:00100000:0.0:1785220581.804903:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519946880 00010000:00010000:0.0:1785220581.804912:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000631cb17a/0xf62499ab9278a6d7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc65 expref: 302 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.804924:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100305 00000100:00100000:0.0:1785220581.804930:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+265:2407:x1871938519945856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.804940:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.804944:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000215d0023/0xf62499ab9278d29e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e118 expref: 265 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.804954:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000215d0023/0xf62499ab9278d29e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e118 expref: 265 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.804971:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000215d0023/0xf62499ab9278d29e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e118 expref: 265 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.804984:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7266:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.804988:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7266:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.804993:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805002:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805019:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.805044:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+264:2407:x1871938519945856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (978us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.805050:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100305 00000100:00100000:0.0:1785220581.805063:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100306 00000100:00100000:0.0:1785220581.805068:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+263:2407:x1871938519946112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.805074:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.805079:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005291ffe6/0xf62499ab9278d09f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e03f expref: 263 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805237:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005291ffe6/0xf62499ab9278d09f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e03f expref: 263 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805251:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005291ffe6/0xf62499ab9278d09f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e03f expref: 263 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805260:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7264:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805263:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7264:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805267:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805275:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805292:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.805308:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+262:2407:x1871938519946112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 240us (765us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.805314:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100306 00000100:00100000:0.0:1785220581.805326:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100307 00000100:00100000:0.0:1785220581.805331:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+261:2407:x1871938519946368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.805338:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.805343:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cd2633e0/0xf62499ab9278d385 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e188 expref: 261 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805352:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cd2633e0/0xf62499ab9278d385 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e188 expref: 261 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805364:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd2633e0/0xf62499ab9278d385 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e188 expref: 261 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805374:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7267:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805377:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7267:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805381:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805395:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805408:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.805422:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+260:2407:x1871938519946368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (863us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.805428:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100307 00000100:00100000:0.0:1785220581.805438:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100308 00000100:00100000:0.0:1785220581.805443:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce79c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+302:2407:x1871938519946624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.805450:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.805455:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009667deb7/0xf62499ab9278a414 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb15 expref: 302 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805465:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009667deb7/0xf62499ab9278a414 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb15 expref: 302 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805479:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009667deb7/0xf62499ab9278a414 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cb15 expref: 302 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805488:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7236:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805491:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7236:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805496:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805508:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805522:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.805535:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce79c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+301:2407:x1871938519946624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (959us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.805541:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100308 00000100:00100000:0.0:1785220581.805551:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100309 00000100:00100000:0.0:1785220581.805556:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+300:2404:x1871938519946880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.805562:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.805567:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000631cb17a/0xf62499ab9278a6d7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc65 expref: 300 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805576:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000631cb17a/0xf62499ab9278a6d7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc65 expref: 300 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805588:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000631cb17a/0xf62499ab9278a6d7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cc65 expref: 300 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805597:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7238:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805600:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7238:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.805604:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805617:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805629:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.805642:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+299:2404:x1871938519946880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (969us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.805647:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100309 00010000:00010000:0.0:1785220581.805667:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805676:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805685:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805693:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.805706:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.806058:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510677760:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.806080:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806148:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806157:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806165:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806172:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806187:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.806272:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+1157:3653:x1871938510676352:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16849us (17253us total) trans 4295070007 rc 0/0 00000100:00100000:3.0:1785220581.806280:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284377 00000100:00100000:3.0:1785220581.806294:0:14161:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510678400 00000100:00100000:3.0:1785220581.806299:0:14161:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510678400 00000100:00100000:3.0:1785220581.806323:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284384 00000100:00100000:3.0:1785220581.806328:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+1220:3652:x1871938510678400:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.806346:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29290 count 1 00010000:00010000:3.0:1785220581.806363:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806376:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004610b86f/0xf62499ab9278d6d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e302 expref: 259 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806384:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x726a:0x0].0x0 since=23s v=(0 0) 00010000:00010000:3.0:1785220581.806387:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.806391:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806413:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004610b86f/0xf62499ab9278d6d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e302 expref: 259 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.806425:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004610b86f/0xf62499ab9278d6d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e302 expref: 259 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.806438:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510678528:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.806476:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.806648:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1220:3652:x1871938510674432:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19096us (20945us total) trans 4295070592 rc 0/0 00000100:00100000:0.0:1785220581.806655:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284369 00010000:00010000:0.0:1785220581.806677:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806690:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806698:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806705:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806719:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.806811:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1220:3654:x1871938510674560:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18323us (21016us total) trans 4295070593 rc 0/0 00000100:00100000:0.0:1785220581.806817:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284371 00000100:00100000:0.0:1785220581.806862:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510676480:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.806877:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.806893:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100304 00010000:00010000:0.0:1785220581.806900:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806908:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806917:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806925:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.806933:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:1.0:1785220581.806944:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510678016:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.806958:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.806974:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.806980:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.806987:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.806991:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.806999:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1220:3652:x1871938510674944:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19240us (21267us total) trans 4295070594 rc 0/0 00010000:00010000:1.0:1785220581.807000:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.807005:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284370 00010000:00010000:1.0:1785220581.807016:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807028:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a4ec0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.807029:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.807038:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.807045:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807046:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.807050:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.807053:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807057:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.807060:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.807061:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:1.0:1785220581.807067:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807076:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807097:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a4e9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807111:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.807114:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807120:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.807124:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807190:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807201:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807208:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a4ed5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7236:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807226:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.807230:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807236:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.807239:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807246:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807256:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807262:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a4eab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807275:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.807278:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807284:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.807287:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807294:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807303:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807311:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a4ea4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807322:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.807325:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807331:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.807334:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807347:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807356:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.807363:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a4eb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.807368:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1220:3654:x1871938510675072:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18858us (19733us total) trans 4295070595 rc 0/0 00000100:00100000:0.0:1785220581.807374:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284372 00010000:00010000:1.0:1785220581.807376:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.807377:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.807379:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.807386:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.807398:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.807403:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.807425:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.807483:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:3.0:1785220581.807620:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4a680/2, rc 0, transno 4295070592 00000100:00100000:3.0:1785220581.807633:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510674432:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.807642:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4bb80/2, rc 0, transno 4295070594 00000100:00100000:3.0:1785220581.807645:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510674944:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.808158:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.808175:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.808185:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.808192:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.808213:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.808309:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1220:3654:x1871938510674688:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19677us (20594us total) trans 4295070596 rc 0/0 00000100:00100000:3.0:1785220581.808315:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284373 00010000:00010000:1.0:1785220581.809624:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.809633:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.809643:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.809647:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.809656:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.809678:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.809686:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a4eb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.809705:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.809707:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.809709:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.809744:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4aa00/2, rc 0, transno 4295070596 00000100:00100000:1.0:1785220581.809751:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510674688:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.809777:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b49f80/2, rc 0, transno 4295070591 00000100:00100000:1.0:1785220581.809781:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510675200:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.809789:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b48000/2, rc 0, transno 4295070593 00000100:00100000:1.0:1785220581.809792:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510674560:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.809800:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b48380/2, rc 0, transno 4295070595 00000100:00100000:1.0:1785220581.809804:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510675072:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.809853:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.811446:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947136 00000100:00100000:1.0:1785220581.811446:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947392 00000100:00100000:2.0:1785220581.811457:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947136 00000100:00100000:1.0:1785220581.811457:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947392 00010000:00010000:1.0:1785220581.811491:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004c39cec8/0xf62499ab9278a9bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdca expref: 300 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.811492:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ffd56c22/0xf62499ab9278a8f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd61 expref: 300 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.811519:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947008 00000100:00100000:1.0:1785220581.811523:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947008 00010000:00010000:1.0:1785220581.811537:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000023fee525/0xf62499ab9278a05c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c98d expref: 301 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.811607:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100310 00000100:00100000:1.0:1785220581.811612:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100311 00000100:00100000:2.0:1785220581.811617:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+258:2406:x1871938519946496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.811617:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+301:2404:x1871938519947392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.811631:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.811631:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.811636:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004c39cec8/0xf62499ab9278a9bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdca expref: 301 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.811637:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a7f2f1e6/0xf62499ab9278d5ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e26f expref: 258 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811649:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004c39cec8/0xf62499ab9278a9bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdca expref: 301 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.811652:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a7f2f1e6/0xf62499ab9278d5ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e26f expref: 258 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.811672:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f2f1e6/0xf62499ab9278d5ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e26f expref: 258 pid: 14202 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811672:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c39cec8/0xf62499ab9278a9bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdca expref: 301 pid: 14266 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.811688:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7269:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.811688:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.811692:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7269:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.811692:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.811699:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811699:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.811710:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811710:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.811734:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.811734:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.811786:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+300:2404:x1871938519947392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 168us (4303us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.811795:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100311 00000100:00100000:0.0:1785220581.811799:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100312 00000100:00100000:0.0:1785220581.811808:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+299:2404:x1871938519947136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.811810:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100313 00000100:00100000:1.0:1785220581.811816:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+299:2404:x1871938519947008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.811816:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.811820:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ffd56c22/0xf62499ab9278a8f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd61 expref: 299 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811821:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.811825:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000023fee525/0xf62499ab9278a05c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c98d expref: 299 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811885:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000023fee525/0xf62499ab9278a05c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c98d expref: 299 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811899:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000023fee525/0xf62499ab9278a05c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c98d expref: 299 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811909:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7232:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.811912:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7232:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.811916:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811923:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.811941:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.811997:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ffd56c22/0xf62499ab9278a8f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd61 expref: 298 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.812013:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffd56c22/0xf62499ab9278a8f9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cd61 expref: 298 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.812022:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.812025:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.812029:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.812036:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.812053:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.812214:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+257:2406:x1871938519946496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 469us (8131us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.812224:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100310 00000100:00100000:1.0:1785220581.812232:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+297:2404:x1871938519947008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 267us (2232us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.812238:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100313 00010000:00010000:2.0:1785220581.812282:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812293:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812305:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812312:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812326:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.812474:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1157:3655:x1871938510677120:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17306us (20013us total) trans 4295070008 rc 0/0 00000100:00100000:2.0:1785220581.812481:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284379 00010000:00010000:2.0:1785220581.812500:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812507:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812514:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812520:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812531:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.812599:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1220:3653:x1871938510676224:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23287us (23663us total) trans 4295070597 rc 0/0 00000100:00100000:2.0:1785220581.812604:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284376 00010000:00010000:2.0:1785220581.812621:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812633:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812640:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812645:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812657:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.812711:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+1157:3653:x1871938510677376:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17274us (20173us total) trans 4295070009 rc 0/0 00000100:00100000:2.0:1785220581.812717:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284381 00010000:00010000:2.0:1785220581.812733:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.812739:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812745:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.812749:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812769:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812781:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812788:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4ee3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7238:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812805:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.812807:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812812:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.812815:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.812821:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.812944:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.812953:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.812962:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.812965:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947264 00000100:00100000:1.0:1785220581.812968:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947264 00010000:00010000:3.0:1785220581.812970:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.812977:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.812983:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000073a5b953/0xf62499ab9278a7d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ccd5 expref: 297 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.812996:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.813003:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100314 00000100:00100000:1.0:1785220581.813010:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+297:2404:x1871938519947264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.813018:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.813023:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000073a5b953/0xf62499ab9278a7d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ccd5 expref: 297 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813033:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000073a5b953/0xf62499ab9278a7d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ccd5 expref: 297 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813046:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000073a5b953/0xf62499ab9278a7d3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ccd5 expref: 297 pid: 14175 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813057:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7239:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.813060:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7239:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.813065:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.813069:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1157:3655:x1871938510677504:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17690us (20551us total) trans 4295070010 rc 0/0 00010000:00010000:1.0:1785220581.813074:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.813075:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284380 00010000:00010000:1.0:1785220581.813111:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.813198:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93664a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+296:2404:x1871938519947264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 187us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.813205:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100314 00010000:00010000:1.0:1785220581.813224:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813226:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813233:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813234:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a4ece lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813242:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813248:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.813249:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813251:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813257:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.813258:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.813261:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813267:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813277:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813284:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a4ef1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813295:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.813299:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813305:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.813308:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813315:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813324:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.813329:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1157:3652:x1871938510677248:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17713us (20727us total) trans 4295070011 rc 0/0 00010000:00010000:2.0:1785220581.813331:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a4eea lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.813336:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284382 00010000:00010000:1.0:1785220581.813354:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.813356:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.813358:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.813359:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813360:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.813365:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.813365:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.813369:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813376:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813385:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813391:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4ef8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7239:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813403:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.813405:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.813406:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.813453:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510678656:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.813456:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.813485:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.813508:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510678656 00000100:00100000:1.0:1785220581.813512:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510678656 00000100:00100000:2.0:1785220581.813513:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510679168:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.813533:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.813535:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284385 00000100:00100000:1.0:1785220581.813541:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1220:3653:x1871938510678656:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.813560:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29291 count 1 00000004:00080000:2.0:1785220581.813564:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa3480/2, rc 0, transno 4295070009 00000100:00100000:2.0:1785220581.813573:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510677376:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.813578:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.813584:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94910a80/2, rc 0, transno 4295070007 00000100:00100000:2.0:1785220581.813587:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94910a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510676352:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.813591:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000944368d1/0xf62499ab9278d7ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e36b expref: 256 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.813595:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94912300/2, rc 0, transno 4295070597 00000100:00100000:2.0:1785220581.813601:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94912300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510676224:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.813605:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x726b:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.813607:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.813611:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.813612:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220581.813616:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94910e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510679040:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.813632:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000944368d1/0xf62499ab9278d7ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e36b expref: 256 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.813636:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.813645:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000944368d1/0xf62499ab9278d7ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e36b expref: 256 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.813655:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510679168 00000100:00100000:1.0:1785220581.813656:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510679424:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.813658:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510679168 00000100:00100000:2.0:1785220581.813675:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510679040 00000100:00100000:2.0:1785220581.813677:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510679040 00000100:00100000:1.0:1785220581.813691:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.813694:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284386 00000100:00100000:2.0:1785220581.813699:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1222:3653:x1871938510679168:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.813705:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284387 00000100:00100000:1.0:1785220581.813710:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1222:3653:x1871938510679040:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.813719:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29294 count 1 00002000:00080000:2.0:1785220581.813727:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29295 count 1 00010000:00010000:1.0:1785220581.813731:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813737:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813741:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b6aecb41/0xf62499ab9278daf5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e52b expref: 256 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813747:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006442166e/0xf62499ab9278dc1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5be expref: 256 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813748:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x726e:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.813751:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.813754:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x726f:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.813768:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813770:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.813774:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813787:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b6aecb41/0xf62499ab9278daf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e52b expref: 256 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813789:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006442166e/0xf62499ab9278dc1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5be expref: 256 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.813797:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b6aecb41/0xf62499ab9278daf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e52b expref: 256 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.813798:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006442166e/0xf62499ab9278dc1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5be expref: 256 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.813808:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510679680:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.813808:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510679552:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.813905:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.813919:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510678272:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.813936:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.813958:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.813980:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510679936:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.814004:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220581.814032:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa1f80/2, rc 0, transno 4295070011 00000100:00100000:3.0:1785220581.814040:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510677248:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.814353:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.814708:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.814726:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510679936 00000100:00100000:2.0:1785220581.814730:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510679936 00000100:00100000:2.0:1785220581.814775:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284388 00000100:00100000:2.0:1785220581.814782:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1157:3652:x1871938510679936:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.814795:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29246 count 1 00010000:00010000:2.0:1785220581.814810:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.814837:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006f7667c2/0xf62499ab9278ac09 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf05 expref: 295 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.814845:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x723e:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.814849:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.814852:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.814876:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006f7667c2/0xf62499ab9278ac09 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf05 expref: 295 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.814886:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006f7667c2/0xf62499ab9278ac09 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf05 expref: 295 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.814898:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510680192:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.814930:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.814978:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510680064:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.814999:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.815024:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510680064 00000100:00100000:2.0:1785220581.815027:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510680064 00000100:00100000:1.0:1785220581.815075:0:14223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284389 00000100:00100000:1.0:1785220581.815083:0:14223:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+1158:3653:x1871938510680064:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.815116:0:14223:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29247 count 1 00000100:00100000:2.0:1785220581.815217:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510678784:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.815231:0:14223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.815244:0:14223:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f5ec17b3/0xf62499ab9278acf0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf7c expref: 295 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.815246:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.815251:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x723f:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.815254:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.815258:0:14223:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.815267:0:14261:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510678784 00000100:00100000:2.0:1785220581.815270:0:14261:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510678784 00010000:00010000:1.0:1785220581.815281:0:14223:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f5ec17b3/0xf62499ab9278acf0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf7c expref: 295 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.815290:0:14223:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f5ec17b3/0xf62499ab9278acf0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf7c expref: 295 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.815291:0:14261:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284390 00000100:00100000:2.0:1785220581.815296:0:14261:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1223:3654:x1871938510678784:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.815300:0:14223:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0001_UUID:14223:1871938510680448:192.168.201.51@tcp:104: 00002000:00080000:2.0:1785220581.815306:0:14261:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29292 count 1 00010000:00010000:2.0:1785220581.815318:0:14261:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.815327:0:14223:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.815330:0:14261:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005b95462d/0xf62499ab9278d943 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e43d expref: 256 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815337:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x726c:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220581.815340:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.815343:0:14261:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815366:0:14261:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005b95462d/0xf62499ab9278d943 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e43d expref: 256 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815380:0:14261:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005b95462d/0xf62499ab9278d943 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e43d expref: 256 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.815390:0:14261:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0000_UUID:14261:1871938510680576:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.815410:0:14261:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.815426:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510679296:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.815445:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.815462:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510679296 00000100:00100000:2.0:1785220581.815465:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510679296 00000100:00100000:2.0:1785220581.815489:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284391 00000100:00100000:2.0:1785220581.815494:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d935a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1224:3654:x1871938510679296:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.815502:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29296 count 1 00010000:00010000:2.0:1785220581.815512:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815522:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e94b9d46/0xf62499ab9278dce6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e62e expref: 256 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815529:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7270:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220581.815531:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7270:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.815535:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815548:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e94b9d46/0xf62499ab9278dce6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e62e expref: 256 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815557:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e94b9d46/0xf62499ab9278dce6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e62e expref: 256 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.815565:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510680704:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.815582:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.815596:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510680320:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.815614:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.815629:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510680320 00000100:00100000:2.0:1785220581.815632:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510680320 00000100:00100000:2.0:1785220581.815647:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284392 00000100:00100000:2.0:1785220581.815651:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1225:3654:x1871938510680320:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.815658:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29297 count 1 00010000:00010000:2.0:1785220581.815666:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815674:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001a4c3255/0xf62499ab9278dde9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e697 expref: 256 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815678:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7271:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220581.815680:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7271:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.815682:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815692:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001a4c3255/0xf62499ab9278dde9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e697 expref: 256 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.815698:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001a4c3255/0xf62499ab9278dde9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e697 expref: 256 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.815704:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0000_UUID:14248:1871938510680832:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.815721:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.815739:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1785220581.815745:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94910380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510678912:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.815779:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.815799:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510678912 00000100:00100000:2.0:1785220581.815803:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510678912 00000100:00100000:0.0:1785220581.815966:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9991b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+295:2404:x1871938519947136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 4158us (8539us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.815974:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100312 00000100:00100000:0.0:1785220581.815998:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284393 00000100:00100000:0.0:1785220581.816004:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1226:3654:x1871938510678912:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.816013:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29293 count 1 00010000:00010000:0.0:1785220581.816022:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.816030:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008dbc24c4/0xf62499ab9278d9eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4ad expref: 256 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.816034:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x726d:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.816036:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.816038:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.816050:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008dbc24c4/0xf62499ab9278d9eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4ad expref: 256 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.816055:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008dbc24c4/0xf62499ab9278d9eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4ad expref: 256 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.816061:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510680960:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.816077:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.816127:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519946752 00000100:00100000:0.0:1785220581.816132:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519946752 00010000:00010000:0.0:1785220581.816154:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b7809/0xf62499ab9278a59c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cbb6 expref: 295 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.816175:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947520 00000100:00100000:0.0:1785220581.816179:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947520 00010000:00010000:0.0:1785220581.816192:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006b08d1da/0xf62499ab9278aa11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce10 expref: 296 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.816209:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947648 00000100:00100000:0.0:1785220581.816212:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947648 00010000:00010000:0.0:1785220581.816224:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004610b86f/0xf62499ab9278d6d4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e302 expref: 257 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.817502:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510679808:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.817549:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.817733:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100315 00000100:00100000:0.0:1785220581.817744:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+296:2406:x1871938519946752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.817768:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.817776:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c37b7809/0xf62499ab9278a59c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cbb6 expref: 296 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.817790:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c37b7809/0xf62499ab9278a59c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cbb6 expref: 296 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.817796:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510679808 00000100:00100000:1.0:1785220581.817802:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510679808 00010000:00010000:0.0:1785220581.817811:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b7809/0xf62499ab9278a59c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cbb6 expref: 296 pid: 14191 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.817838:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7237:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.817842:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7237:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.817846:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284394 00010000:00010000:0.0:1785220581.817848:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.817853:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1159:3654:x1871938510679808:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.817857:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220581.817869:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29245 count 1 00010000:00010000:0.0:1785220581.817878:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.817887:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.817900:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000be959df5/0xf62499ab9278ab06 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce79 expref: 295 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.817900:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+295:2406:x1871938519946752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 159us (4443us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.817907:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x723d:0x0].0x0 since=26s v=(0 0) 00000100:00100000:0.0:1785220581.817907:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100315 00010000:00010000:1.0:1785220581.817910:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.817914:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.817925:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100316 00000100:00100000:0.0:1785220581.817930:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+294:2407:x1871938519947520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.817936:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000be959df5/0xf62499ab9278ab06 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce79 expref: 294 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.817937:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.817942:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006b08d1da/0xf62499ab9278aa11 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce10 expref: 294 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.817947:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000be959df5/0xf62499ab9278ab06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce79 expref: 294 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.817951:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006b08d1da/0xf62499ab9278aa11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce10 expref: 294 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.817958:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510681088:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.817963:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b08d1da/0xf62499ab9278aa11 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce10 expref: 294 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.817972:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.817975:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.817978:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.817985:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.817991:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f180 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.818011:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.818025:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.818026:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+293:2407:x1871938519947520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (4068us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.818033:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100316 00010000:00010000:1.0:1785220581.818034:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.818043:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.818050:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.818062:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.818070:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100317 00000100:00100000:0.0:1785220581.818076:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+257:2406:x1871938519947648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.818083:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.818114:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004610b86f/0xf62499ab9278d6d4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e302 expref: 257 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818124:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004610b86f/0xf62499ab9278d6d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e302 expref: 257 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818138:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.818141:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.818144:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818151:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818162:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.818175:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99066300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+257:2406:x1871938519947648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (3822us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.818183:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100317 00010000:00010000:0.0:1785220581.818206:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818214:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818222:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.818230:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1160:3654:x1871938510678144:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15674us (15849us total) trans 4295070012 rc 0/0 00010000:00010000:0.0:1785220581.818230:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.818237:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284383 00010000:00010000:0.0:1785220581.818241:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.818326:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1160:3653:x1871938510673536:12345-0@lo:6:osp-syn-1-0.0 Request processed in 29306us (29550us total) trans 4295070013 rc 0/0 00000100:00100000:0.0:1785220581.818333:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284375 00010000:00010000:0.0:1785220581.818430:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.818440:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818448:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.818452:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818464:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818477:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818483:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4eff lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818495:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.818497:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818501:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.818503:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818507:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818513:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818517:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4ec7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7232:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.818524:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.818525:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.818527:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.818567:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb65500/2, rc 0, transno 4295070010 00000100:00100000:0.0:1785220581.818579:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510677504:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.818592:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa3100/2, rc 0, transno 4295070008 00000100:00100000:0.0:1785220581.818596:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510677120:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.818783:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99064a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510681216:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.818816:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.818927:0:14161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004610b86f/0xf62499ab9278d6d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e302 expref: 256 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.818939:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510678528:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.818958:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.818962:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510681216 00010000:00010000:3.0:1785220581.818966:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.818967:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510681216 00010000:00010000:3.0:1785220581.818973:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.818982:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.818989:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.819001:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.819004:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284395 00000100:00100000:2.0:1785220581.819010:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1160:3655:x1871938510681216:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.819025:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29248 count 1 00010000:00010000:2.0:1785220581.819038:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819050:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c0708716/0xf62499ab9278adb4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfd0 expref: 292 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819058:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7240:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.819061:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7240:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.819065:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819174:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c0708716/0xf62499ab9278adb4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfd0 expref: 292 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819186:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c0708716/0xf62499ab9278adb4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfd0 expref: 292 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.819197:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510681472:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.819231:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.819247:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819255:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819263:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819270:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819280:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.819294:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+1227:3652:x1871938510678400:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12965us (15442us total) trans 4295070598 rc 0/0 00000100:00100000:3.0:1785220581.819301:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284384 00000100:00100000:3.0:1785220581.819330:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510681344:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.819352:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.819354:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1161:3654:x1871938510676864:12345-0@lo:6:osp-syn-1-0.0 Request processed in 28601us (29023us total) trans 4295070014 rc 0/0 00000100:00100000:2.0:1785220581.819360:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284378 00000004:00080000:3.0:1785220581.819374:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be5880/2, rc 0, transno 4295070598 00000100:00100000:2.0:1785220581.819375:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510681344 00000100:00100000:2.0:1785220581.819378:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510681344 00000100:00100000:3.0:1785220581.819386:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510678400:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.819395:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284396 00000100:00100000:2.0:1785220581.819401:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1161:3652:x1871938510681344:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.819411:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29249 count 1 00010000:00010000:2.0:1785220581.819421:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819432:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d8dc65b9/0xf62499ab9278ae24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d039 expref: 292 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819443:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7241:0x0].0x0 since=26s v=(0 0) 00010000:00010000:2.0:1785220581.819446:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7241:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.819449:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819466:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d8dc65b9/0xf62499ab9278ae24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d039 expref: 292 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819476:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d8dc65b9/0xf62499ab9278ae24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d039 expref: 292 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.819485:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510681600:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.819503:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.819514:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.819519:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819527:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.819531:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819539:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819560:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819567:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a4f06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819585:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.819588:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819595:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.819598:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819604:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819613:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819619:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4edc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7237:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.819633:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.819634:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.819637:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.819644:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.819692:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510681728:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.819719:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.819741:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510681728 00000100:00100000:2.0:1785220581.819744:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510681728 00000100:00100000:2.0:1785220581.819778:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284397 00000100:00100000:2.0:1785220581.819785:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1227:3653:x1871938510681728:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.819809:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29298 count 1 00010000:00010000:2.0:1785220581.819822:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.820243:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.820881:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510679424:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.820904:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.820937:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510679552:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.820949:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.820964:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.821169:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b49c00/2, rc 0, transno 4295070012 00000100:00100000:0.0:1785220581.821183:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510678144:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.821195:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94912680/2, rc 0, transno 4295070014 00000100:00100000:0.0:1785220581.821199:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94912680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510676864:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.821366:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb21f80/2, rc 0, transno 4295070013 00000100:00100000:3.0:1785220581.821372:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510673536:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.821404:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510682112:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.821408:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510681984:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.821442:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.821487:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510681984 00000100:00100000:3.0:1785220581.821493:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510681984 00000100:00100000:3.0:1785220581.821527:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284398 00000100:00100000:3.0:1785220581.821535:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1162:3655:x1871938510681984:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.821553:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29251 count 1 00010000:00010000:3.0:1785220581.821572:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.821590:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd263614/0xf62499ab9278b00e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d112 expref: 292 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.821598:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7243:0x0].0x0 since=26s v=(0 0) 00010000:00010000:3.0:1785220581.821602:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7243:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.821607:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.821634:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd263614/0xf62499ab9278b00e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d112 expref: 292 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.821647:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd263614/0xf62499ab9278b00e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d112 expref: 292 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.821660:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510682240:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.821711:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.822660:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.822670:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510681856:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.822707:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.822753:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.822793:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510682112 00000100:00100000:1.0:1785220581.822793:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510681856 00000100:00100000:3.0:1785220581.822798:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510682112 00000100:00100000:1.0:1785220581.822798:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510681856 00000100:00100000:1.0:1785220581.822893:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284399 00000100:00100000:1.0:1785220581.822902:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1164:3655:x1871938510681856:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.822920:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29250 count 1 00010000:00010000:1.0:1785220581.822938:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.822956:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dd1ecba1/0xf62499ab9278af04 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d08d expref: 292 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.822964:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7242:0x0].0x0 since=26s v=(0 0) 00010000:00010000:1.0:1785220581.822968:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7242:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.822973:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.823000:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dd1ecba1/0xf62499ab9278af04 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d08d expref: 292 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.823010:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dd1ecba1/0xf62499ab9278af04 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d08d expref: 292 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.823022:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510682368:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.823059:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.823072:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284400 00000100:00100000:3.0:1785220581.823081:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1164:3652:x1871938510682112:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.823195:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000296f95c9/0xf62499ab9278dede lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e71c expref: 255 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.823203:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7272:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220581.823206:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7272:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.823209:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.823230:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000296f95c9/0xf62499ab9278dede lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e71c expref: 255 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.823241:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000296f95c9/0xf62499ab9278dede lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e71c expref: 255 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.823252:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510682496:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.823279:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df180 going to sleep for 11 seconds 00002000:00080000:3.0:1785220581.823305:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29252 count 1 00010000:00010000:3.0:1785220581.823319:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.823331:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001e12130d/0xf62499ab9278b0ee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d190 expref: 292 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.823339:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7244:0x0].0x0 since=26s v=(0 0) 00010000:00010000:3.0:1785220581.823341:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7244:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.823345:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.823363:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001e12130d/0xf62499ab9278b0ee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d190 expref: 292 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.823372:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001e12130d/0xf62499ab9278b0ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d190 expref: 292 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.823382:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510682624:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.823407:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.824279:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.824299:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.824421:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947776 00000100:00100000:1.0:1785220581.824427:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947776 00010000:00010000:1.0:1785220581.824454:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000944368d1/0xf62499ab9278d7ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e36b expref: 256 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.824479:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519947904 00000100:00100000:1.0:1785220581.824483:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519947904 00010000:00010000:1.0:1785220581.824496:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aecb41/0xf62499ab9278daf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e52b expref: 257 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.824517:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100318 00000100:00100000:1.0:1785220581.824524:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+257:2407:x1871938519947776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.824556:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.824563:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000944368d1/0xf62499ab9278d7ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e36b expref: 257 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824580:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000944368d1/0xf62499ab9278d7ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e36b expref: 257 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824601:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944368d1/0xf62499ab9278d7ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e36b expref: 257 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824615:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.824618:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.824624:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824632:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824664:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.824696:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+256:2407:x1871938519947776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 172us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.824703:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100318 00000100:00100000:1.0:1785220581.824720:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100319 00000100:00100000:1.0:1785220581.824727:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+255:2406:x1871938519947904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.824734:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.824739:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b6aecb41/0xf62499ab9278daf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e52b expref: 255 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824754:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b6aecb41/0xf62499ab9278daf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e52b expref: 255 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824777:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aecb41/0xf62499ab9278daf5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e52b expref: 255 pid: 14210 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824788:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.824792:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726e:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.824793:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.824796:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.824803:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.824809:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.824820:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.824826:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.824844:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.824859:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.824873:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.824888:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.824905:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.824916:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510679680:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.824938:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.824969:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510680192:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.824982:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825005:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0000_UUID:14261:1871938510680576:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.825021:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825044:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510680704:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.825054:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825077:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0000_UUID:14248:1871938510680832:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.825123:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825142:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948288 00000100:00100000:2.0:1785220581.825145:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948288 00010000:00010000:2.0:1785220581.825162:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f5ec17b3/0xf62499ab9278acf0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf7c expref: 293 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825186:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948800 00000100:00100000:2.0:1785220581.825189:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948800 00010000:00010000:2.0:1785220581.825201:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008dbc24c4/0xf62499ab9278d9eb lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4ad expref: 255 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825219:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948416 00000100:00100000:2.0:1785220581.825222:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948416 00000100:00100000:3.0:1785220581.825230:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948032 00010000:00010000:2.0:1785220581.825233:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005b95462d/0xf62499ab9278d943 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e43d expref: 256 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.825234:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948032 00010000:00010000:3.0:1785220581.825249:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006442166e/0xf62499ab9278dc1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5be expref: 257 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825249:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948544 00000100:00100000:2.0:1785220581.825251:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948544 00010000:00010000:2.0:1785220581.825261:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e94b9d46/0xf62499ab9278dce6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e62e expref: 258 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.825262:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948928 00000100:00100000:3.0:1785220581.825264:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948928 00000100:00100000:2.0:1785220581.825271:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948160 00000100:00100000:2.0:1785220581.825273:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948160 00010000:00010000:3.0:1785220581.825274:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000be959df5/0xf62499ab9278ab06 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce79 expref: 294 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.825282:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006f7667c2/0xf62499ab9278ac09 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf05 expref: 295 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.825284:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519948672 00000100:00100000:3.0:1785220581.825286:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519948672 00000100:00100000:2.0:1785220581.825295:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100320 00010000:00010000:3.0:1785220581.825296:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001a4c3255/0xf62499ab9278dde9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e697 expref: 259 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.825301:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+295:2405:x1871938519948288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.825307:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100321 00010000:00010000:2.0:1785220581.825309:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220581.825313:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+259:2405:x1871938519948800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.825313:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f5ec17b3/0xf62499ab9278acf0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf7c expref: 295 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825319:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.825323:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008dbc24c4/0xf62499ab9278d9eb lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4ad expref: 259 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.825328:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f5ec17b3/0xf62499ab9278acf0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf7c expref: 295 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825332:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008dbc24c4/0xf62499ab9278d9eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4ad expref: 259 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825343:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.825343:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825347:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.825347:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825352:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.825352:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.825361:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825365:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.825372:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.825375:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.825397:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+259:2405:x1871938519948800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (587us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.825403:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100321 00000100:00100000:3.0:1785220581.825418:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100322 00000100:00100000:3.0:1785220581.825424:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+258:2405:x1871938519948416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.825431:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.825435:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005b95462d/0xf62499ab9278d943 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e43d expref: 258 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825444:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005b95462d/0xf62499ab9278d943 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e43d expref: 258 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825456:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b95462d/0xf62499ab9278d943 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e43d expref: 258 pid: 13909 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825466:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825469:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825473:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825484:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825499:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.825563:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+295:2405:x1871938519948288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 260us (768us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.825574:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100320 00000100:00100000:3.0:1785220581.825583:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+257:2405:x1871938519948416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 158us (756us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.825588:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100323 00000100:00100000:3.0:1785220581.825628:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100322 00000100:00100000:1.0:1785220581.825640:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+256:2406:x1871938519947904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 914us (1340us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.825642:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100324 00000100:00100000:1.0:1785220581.825646:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100319 00000100:00100000:3.0:1785220581.825648:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+256:2407:x1871938519948544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.825657:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220581.825659:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100325 00010000:00010000:3.0:1785220581.825662:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e94b9d46/0xf62499ab9278dce6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e62e expref: 255 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.825665:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+294:2407:x1871938519948928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.825672:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.825673:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e94b9d46/0xf62499ab9278dce6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e62e expref: 255 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.825676:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000be959df5/0xf62499ab9278ab06 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce79 expref: 294 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825685:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e94b9d46/0xf62499ab9278dce6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e62e expref: 255 pid: 14256 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.825691:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000be959df5/0xf62499ab9278ab06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce79 expref: 294 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825695:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7270:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825698:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7270:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825701:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.825702:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.825705:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.825709:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825715:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.825717:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.825727:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.825730:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.825743:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+254:2407:x1871938519948544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (883us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.825743:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+294:2407:x1871938519948928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (868us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.825749:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100325 00000100:00100000:2.0:1785220581.825773:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+254:2407:x1871938519948032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.825781:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.825787:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006442166e/0xf62499ab9278dc1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5be expref: 254 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.825798:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100324 00010000:00010000:2.0:1785220581.825800:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006442166e/0xf62499ab9278dc1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5be expref: 254 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.825811:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100326 00000100:00100000:3.0:1785220581.825818:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+294:2407:x1871938519948160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.825819:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006442166e/0xf62499ab9278dc1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5be expref: 253 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825849:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.825855:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006f7667c2/0xf62499ab9278ac09 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf05 expref: 294 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825864:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006f7667c2/0xf62499ab9278ac09 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf05 expref: 294 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825878:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f7667c2/0xf62499ab9278ac09 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf05 expref: 294 pid: 14173 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825888:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825891:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x723e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.825895:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.825898:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.825901:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x726f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.825906:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825908:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.825914:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.825925:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.825927:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.826009:0:14223:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f5ec17b3/0xf62499ab9278acf0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cf7c expref: 293 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.826026:0:14223:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0001_UUID:14223:1871938510680448:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.826032:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+292:2407:x1871938519948160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 215us (1142us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.826038:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100326 00010000:00010000:0.0:1785220581.826041:0:14223:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826049:0:14223:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.826050:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100327 00000100:00100000:3.0:1785220581.826056:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+252:2407:x1871938519948672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.826057:0:14223:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.826063:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.826068:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001a4c3255/0xf62499ab9278dde9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e697 expref: 252 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826072:0:14223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.826077:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001a4c3255/0xf62499ab9278dde9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e697 expref: 252 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826080:0:14223:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.826106:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a4c3255/0xf62499ab9278dde9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e697 expref: 252 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.826116:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7271:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.826119:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7271:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.826121:0:14223:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.826123:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.826145:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.826160:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.826174:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+251:2407:x1871938519948672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 118us (1268us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.826179:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100327 00000100:00100000:0.0:1785220581.826303:0:14223:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+1165:3653:x1871938510680064:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11217us (11299us total) trans 4295070015 rc 0/0 00000100:00100000:0.0:1785220581.826314:0:14223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284389 00010000:00010000:0.0:1785220581.826351:0:14208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008dbc24c4/0xf62499ab9278d9eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4ad expref: 251 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.826365:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510680960:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.826380:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826391:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826401:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826410:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826421:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.826437:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.826581:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+249:2407:x1871938519948032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 349us (1097us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.826591:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100323 00010000:00010000:2.0:1785220581.826637:0:14261:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.826649:0:14261:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.826658:0:14261:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.826668:0:14261:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.826685:0:14261:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.826794:0:14261:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1228:3654:x1871938510678784:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11497us (11546us total) trans 4295070599 rc 0/0 00000100:00100000:2.0:1785220581.826805:0:14261:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284390 00010000:00010000:1.0:1785220581.826923:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.826936:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.826945:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.826954:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.826969:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:1.0:1785220581.827058:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1228:3653:x1871938510678656:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13514us (13570us total) trans 4295070600 rc 0/0 00000100:00100000:1.0:1785220581.827065:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284385 00010000:00010000:2.0:1785220581.827158:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.827168:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.827176:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.827183:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.827192:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00010000:00010000:1.0:1785220581.827213:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827225:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827233:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827240:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827251:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:1.0:1785220581.827328:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1228:3653:x1871938510679040:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13617us (13690us total) trans 4295070601 rc 0/0 00000100:00100000:1.0:1785220581.827335:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284387 00010000:00010000:1.0:1785220581.827372:0:14251:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000be959df5/0xf62499ab9278ab06 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce79 expref: 290 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.827387:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510681088:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.827405:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827414:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827424:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827432:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827442:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.827451:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/7 locks to regular blp list, 7 blwis in pool 00000100:00100000:2.0:1785220581.827555:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d935a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1228:3654:x1871938510679296:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12057us (12105us total) trans 4295070602 rc 0/0 00000100:00100000:2.0:1785220581.827562:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284391 00010000:00010000:2.0:1785220581.827609:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.827725:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1228:3654:x1871938510678912:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11720us (11943us total) trans 4295070603 rc 0/0 00000100:00100000:0.0:1785220581.827735:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284393 00010000:00010000:0.0:1785220581.827773:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 6 total in blp. (6 blwis in pool) 00010000:00010000:0.0:1785220581.827783:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.827792:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.827798:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.827808:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.827898:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.827915:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.827923:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.827964:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1165:3654:x1871938510679808:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10111us (10414us total) trans 4295070016 rc 0/0 00000100:00100000:1.0:1785220581.827974:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284394 00010000:00010000:1.0:1785220581.827995:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:1.0:1785220581.828001:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828006:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.828008:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828017:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828026:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828030:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a4f45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828042:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.828045:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828049:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.828050:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828054:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828059:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828063:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a4f30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828074:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.828076:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828080:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.828082:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828187:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828199:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828206:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a4f0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828219:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.828222:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828228:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.828232:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.828243:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510681472:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.828244:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828253:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.828257:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828260:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a4f37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828272:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.828275:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.828282:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510681600:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.828282:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.828285:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.828293:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828293:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828303:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828310:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a4f14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828325:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.828328:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828335:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.828338:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828345:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828352:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828357:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a4f4c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.828366:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.828367:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.828368:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.828392:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083e300/2, rc 0, transno 4295070015 00000100:00100000:1.0:1785220581.828402:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510680064:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.828409:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94910e00/2, rc 0, transno 4295070601 00000100:00100000:1.0:1785220581.828423:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94910e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510679040:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.828434:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94911c00/2, rc 0, transno 4295070600 00000100:00100000:1.0:1785220581.828438:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510678656:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.828610:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510682752:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.828646:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.828712:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510683008:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.828736:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.828773:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.828779:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510682880:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.828801:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.828809:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510682752 00000100:00100000:3.0:1785220581.828814:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510682752 00010000:00010000:0.0:1785220581.828901:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.828911:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a4f29 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.828926:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.828928:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.828935:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.829433:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.829562:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1165:3652:x1871938510679936:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14779us (15555us total) trans 4295070017 rc 0/0 00000100:00100000:2.0:1785220581.829570:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284388 00000100:00100000:2.0:1785220581.829588:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510683008 00000100:00100000:2.0:1785220581.829592:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510683008 00010000:00010000:1.0:1785220581.829632:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.829638:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.829646:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.829649:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.829658:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.829671:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.829677:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a4f22 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x723e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.829692:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.829694:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.829696:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.829721:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083dc00/2, rc 0, transno 4295070017 00000100:00100000:1.0:1785220581.829731:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510679936:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.829784:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.829793:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510683136:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.829819:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.831580:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510682880 00000100:00100000:3.0:1785220581.831590:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510682880 00000100:00100000:3.0:1785220581.831627:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510683136 00000100:00100000:3.0:1785220581.831631:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510683136 00000100:00100000:3.0:1785220581.831652:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284401 00000100:00100000:0.0:1785220581.831659:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284402 00000100:00100000:3.0:1785220581.831661:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1229:3653:x1871938510682752:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.831668:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1166:3655:x1871938510683008:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.831689:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29299 count 1 00002000:00080000:0.0:1785220581.831689:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29253 count 1 00010000:00010000:3.0:1785220581.831711:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.831711:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.831732:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d7566df1/0xf62499ab9278e169 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e82d expref: 248 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.831732:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000757bdf8d/0xf62499ab9278b1ab lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1f2 expref: 289 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.831742:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7273:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.831742:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7245:0x0].0x0 since=26s v=(0 0) 00010000:00010000:3.0:1785220581.831746:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7273:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.831746:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7245:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.831752:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.831752:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.831794:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d7566df1/0xf62499ab9278e169 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e82d expref: 248 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.831794:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000757bdf8d/0xf62499ab9278b1ab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1f2 expref: 289 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.831809:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d7566df1/0xf62499ab9278e169 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e82d expref: 248 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.831809:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000757bdf8d/0xf62499ab9278b1ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1f2 expref: 289 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.831823:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510683264:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.831823:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510683392:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.831961:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.832004:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94910380/2, rc 0, transno 4295070603 00000100:00100000:0.0:1785220581.832026:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94910380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510678912:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.832044:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083f100/2, rc 0, transno 4295070016 00000100:00100000:0.0:1785220581.832055:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510679808:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.832133:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510683520:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.832168:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.832191:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510683520 00000100:00100000:0.0:1785220581.832195:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510683520 00000100:00100000:0.0:1785220581.832220:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284403 00000100:00100000:0.0:1785220581.832227:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1229:3655:x1871938510682880:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.832240:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29300 count 1 00010000:00010000:0.0:1785220581.832252:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832264:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b0fafecc/0xf62499ab9278e249 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8ab expref: 248 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832271:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7274:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.832274:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7274:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.832278:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832298:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b0fafecc/0xf62499ab9278e249 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8ab expref: 248 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832307:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b0fafecc/0xf62499ab9278e249 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8ab expref: 248 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.832323:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510683648:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.832346:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.832362:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284404 00000100:00100000:0.0:1785220581.832369:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1167:3653:x1871938510683136:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.832381:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29254 count 1 00010000:00010000:0.0:1785220581.832393:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832404:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006a19e89c/0xf62499ab9278b364 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2b6 expref: 289 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832411:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7246:0x0].0x0 since=26s v=(0 0) 00010000:00010000:0.0:1785220581.832414:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7246:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.832417:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832435:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006a19e89c/0xf62499ab9278b364 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2b6 expref: 289 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.832444:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006a19e89c/0xf62499ab9278b364 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2b6 expref: 289 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.832454:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510683776:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.832466:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.832476:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.832483:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284405 00000100:00100000:3.0:1785220581.832491:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1167:3653:x1871938510683520:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.832503:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29255 count 1 00000004:00080000:0.0:1785220581.832508:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94911500/2, rc 0, transno 4295070599 00000100:00100000:0.0:1785220581.832512:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510678784:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.832515:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.832521:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991a300/2, rc 0, transno 4295070602 00000100:00100000:0.0:1785220581.832523:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510679296:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.832527:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ae95739e/0xf62499ab9278b444 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d31f expref: 289 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832538:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7247:0x0].0x0 since=25s v=(0 0) 00010000:00010000:3.0:1785220581.832541:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7247:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.832545:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832565:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ae95739e/0xf62499ab9278b444 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d31f expref: 289 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832574:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ae95739e/0xf62499ab9278b444 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d31f expref: 289 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.832583:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510683904:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.832606:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.832723:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.832737:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0001_UUID:14144:1871938510682240:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.832741:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949184 00000100:00100000:3.0:1785220581.832744:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949184 00010000:00010000:0.0:1785220581.832754:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832772:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c0708716/0xf62499ab9278adb4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfd0 expref: 290 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.832796:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100328 00000100:00100000:3.0:1785220581.832803:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2407:x1871938519949184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.832811:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.832816:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c0708716/0xf62499ab9278adb4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfd0 expref: 290 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832887:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c0708716/0xf62499ab9278adb4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfd0 expref: 290 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.832904:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510682368:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.832908:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0708716/0xf62499ab9278adb4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfd0 expref: 290 pid: 14259 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.832918:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832920:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7240:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.832923:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7240:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.832928:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832936:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.832954:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.832977:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+289:2407:x1871938519949184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 174us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.832984:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100328 00000100:00100000:1.0:1785220581.833030:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.833047:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949440 00000100:00100000:1.0:1785220581.833050:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949440 00010000:00010000:1.0:1785220581.833065:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dd1ecba1/0xf62499ab9278af04 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d08d expref: 289 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.833225:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100329 00000100:00100000:1.0:1785220581.833235:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+289:2407:x1871938519949440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.833243:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.833248:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dd1ecba1/0xf62499ab9278af04 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d08d expref: 289 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.833258:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dd1ecba1/0xf62499ab9278af04 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d08d expref: 289 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.833277:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dd1ecba1/0xf62499ab9278af04 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d08d expref: 289 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.833288:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7242:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.833291:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7242:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.833295:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.833303:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.833320:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.833339:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+288:2407:x1871938519949440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.833766:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.833792:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.833845:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510682624:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.833861:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.833897:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.833919:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.833935:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949056 00000100:00100000:3.0:1785220581.833940:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949056 00010000:00010000:3.0:1785220581.833957:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d8dc65b9/0xf62499ab9278ae24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d039 expref: 289 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.833972:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.833985:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.833994:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.834002:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.834020:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.834194:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1230:3653:x1871938510679168:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20494us (20659us total) trans 4295070604 rc 0/0 00000100:00100000:2.0:1785220581.834202:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284386 00010000:00010000:2.0:1785220581.834303:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.834313:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.834321:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.834328:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.834341:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.834409:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1230:3654:x1871938510680320:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18756us (18792us total) trans 4295070605 rc 0/0 00000100:00100000:2.0:1785220581.834416:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284392 00000100:00100000:2.0:1785220581.834452:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949312 00000100:00100000:2.0:1785220581.834455:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949312 00010000:00010000:2.0:1785220581.834470:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd263614/0xf62499ab9278b00e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d112 expref: 290 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.835105:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949568 00000100:00100000:0.0:1785220581.835110:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949568 00010000:00010000:0.0:1785220581.835131:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000296f95c9/0xf62499ab9278dede lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e71c expref: 249 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.835157:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949696 00000100:00100000:0.0:1785220581.835160:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949696 00010000:00010000:0.0:1785220581.835175:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001e12130d/0xf62499ab9278b0ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d190 expref: 291 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.835198:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100330 00000100:00100000:0.0:1785220581.835204:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+249:2405:x1871938519949568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.835216:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.835222:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000296f95c9/0xf62499ab9278dede lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e71c expref: 249 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835238:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000296f95c9/0xf62499ab9278dede lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e71c expref: 249 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835259:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7272:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.835264:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7272:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.835269:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835279:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835297:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.835332:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+249:2405:x1871938519949568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 127us (1433us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.835339:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100330 00000100:00100000:0.0:1785220581.835353:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100331 00000100:00100000:0.0:1785220581.835359:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+291:2405:x1871938519949696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.835368:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.835372:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001e12130d/0xf62499ab9278b0ee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d190 expref: 291 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835382:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001e12130d/0xf62499ab9278b0ee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d190 expref: 291 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835401:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001e12130d/0xf62499ab9278b0ee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d190 expref: 291 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835414:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7244:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.835417:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7244:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.835421:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835429:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835444:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.835462:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+290:2405:x1871938519949696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (1542us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.835468:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100331 00010000:00010000:0.0:1785220581.835490:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835499:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835506:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835513:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835529:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.835639:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1168:3655:x1871938510681216:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16628us (16821us total) trans 4295070018 rc 0/0 00000100:00100000:0.0:1785220581.835647:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284395 00010000:00010000:0.0:1785220581.835674:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.835680:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835689:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.835693:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835704:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835717:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835725:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a4f1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835744:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.835748:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835773:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.835776:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835784:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835794:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835801:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a4f3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835816:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.835819:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835837:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.835841:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835848:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835858:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835866:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a4f53 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.835881:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.835883:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.835886:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.835927:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99919f80/2, rc 0, transno 4295070604 00000100:00100000:0.0:1785220581.835939:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510679168:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.835946:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.835959:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510684288:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.835991:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.836030:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510684032:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.836052:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.836076:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7d180/2, rc 0, transno 4295070605 00000100:00100000:0.0:1785220581.836081:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510680320:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.836106:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220581.836112:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510684160:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.836135:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.836312:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100329 00000100:00100000:1.0:1785220581.836339:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510684288 00000100:00100000:1.0:1785220581.836343:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510684288 00000100:00100000:1.0:1785220581.836376:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510684032 00000100:00100000:1.0:1785220581.836380:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510684032 00000100:00100000:1.0:1785220581.836397:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510684160 00000100:00100000:1.0:1785220581.836400:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510684160 00000100:00100000:1.0:1785220581.836417:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284406 00000100:00100000:1.0:1785220581.836425:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1232:3653:x1871938510684288:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.836447:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29304 count 1 00010000:00010000:1.0:1785220581.836465:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836484:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003dfe8ac8/0xf62499ab9278e528 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea02 expref: 248 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836491:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7278:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.836495:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7278:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.836498:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836525:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003dfe8ac8/0xf62499ab9278e528 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea02 expref: 248 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836538:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003dfe8ac8/0xf62499ab9278e528 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea02 expref: 248 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.836549:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510684416:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.836590:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.836607:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284407 00000100:00100000:1.0:1785220581.836614:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1232:3654:x1871938510684032:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.836626:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29301 count 1 00010000:00010000:1.0:1785220581.836639:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836650:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e3aebb89/0xf62499ab9278e361 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e91b expref: 248 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836661:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7275:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.836664:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7275:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.836666:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100332 00010000:00010000:1.0:1785220581.836668:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.836676:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+288:2407:x1871938519949056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.836685:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.836685:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e3aebb89/0xf62499ab9278e361 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e91b expref: 248 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836691:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d8dc65b9/0xf62499ab9278ae24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d039 expref: 288 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836694:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e3aebb89/0xf62499ab9278e361 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e91b expref: 248 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836702:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d8dc65b9/0xf62499ab9278ae24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d039 expref: 288 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.836703:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510684544:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.836715:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8dc65b9/0xf62499ab9278ae24 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d039 expref: 288 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836726:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7241:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.836727:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f180 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.836729:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7241:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.836734:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836745:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836746:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836753:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.836776:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836779:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.836783:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.836798:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+287:2407:x1871938519949056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 125us (3043us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.836799:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.836805:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100332 00000100:00100000:3.0:1785220581.836914:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284408 00000100:00100000:3.0:1785220581.836922:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1232:3654:x1871938510684160:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.836934:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29303 count 1 00010000:00010000:3.0:1785220581.836946:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836957:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009cabec6c/0xf62499ab9278e480 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e984 expref: 248 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836964:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7277:0x0].0x0 since=23s v=(0 0) 00010000:00010000:3.0:1785220581.836966:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7277:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.836969:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836988:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009cabec6c/0xf62499ab9278e480 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e984 expref: 248 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.836996:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009cabec6c/0xf62499ab9278e480 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e984 expref: 248 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.837006:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510684672:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.837028:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.837043:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837050:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837057:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837065:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837073:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220581.837188:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.837194:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837202:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.837206:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837215:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837227:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837234:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a4f6f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7242:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837248:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.837251:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837257:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.837259:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837266:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837275:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837281:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a4f76 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7244:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837298:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.837300:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.837302:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.837313:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1168:3652:x1871938510682112:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14234us (14559us total) trans 4295070019 rc 0/0 00000100:00100000:3.0:1785220581.837320:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284400 00000004:00080000:0.0:1785220581.837323:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbd500/2, rc 0, transno 4295070019 00000100:00100000:0.0:1785220581.837332:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510682112:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.837340:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837348:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837360:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837366:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837376:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.837379:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99066300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510684800:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.837405:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.837433:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.837437:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.837445:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1168:3652:x1871938510681344:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18044us (18092us total) trans 4295070020 rc 0/0 00010000:00010000:0.0:1785220581.837445:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.837448:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.837452:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284396 00010000:00010000:0.0:1785220581.837456:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.837465:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510684800 00010000:00010000:0.0:1785220581.837467:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.837469:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510684800 00010000:00010000:0.0:1785220581.837474:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a4f5a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7241:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.837486:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.837488:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.837489:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284409 00010000:00010000:0.0:1785220581.837490:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.837494:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1168:3653:x1871938510684800:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.837505:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29256 count 1 00000004:00080000:0.0:1785220581.837505:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99067b80/2, rc 0, transno 4295070020 00000100:00100000:0.0:1785220581.837512:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99067b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510681344:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.837517:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837528:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d039a7a4/0xf62499ab9278b524 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d39d expref: 286 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837536:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7248:0x0].0x0 since=25s v=(0 0) 00010000:00010000:3.0:1785220581.837539:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7248:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.837543:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.837543:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510684928:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.837565:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.837566:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d039a7a4/0xf62499ab9278b524 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d39d expref: 286 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.837575:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d039a7a4/0xf62499ab9278b524 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d39d expref: 286 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.837586:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510685056:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.837614:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.837629:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510684928 00000100:00100000:3.0:1785220581.837632:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510684928 00000100:00100000:3.0:1785220581.837732:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284410 00000100:00100000:3.0:1785220581.837740:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1169:3654:x1871938510684928:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.837752:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29257 count 1 00000100:00100000:1.0:1785220581.837968:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1170:3655:x1871938510681856:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15067us (15259us total) trans 4295070021 rc 0/0 00000100:00100000:1.0:1785220581.837974:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284399 00000100:00100000:1.0:1785220581.838142:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.838161:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949824 00000100:00100000:1.0:1785220581.838165:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949824 00010000:00010000:1.0:1785220581.838183:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d7566df1/0xf62499ab9278e169 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e82d expref: 249 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.838207:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100333 00000100:00100000:1.0:1785220581.838214:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+249:2406:x1871938519949824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.838222:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.838227:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d7566df1/0xf62499ab9278e169 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e82d expref: 249 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.838238:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d7566df1/0xf62499ab9278e169 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e82d expref: 249 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.838253:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7273:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.838256:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7273:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.838261:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.838273:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.838287:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.838306:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+249:2406:x1871938519949824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.838313:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100333 00010000:00010000:3.0:1785220581.838411:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838426:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000072346d69/0xf62499ab9278b5a2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3ea expref: 286 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838432:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7249:0x0].0x0 since=25s v=(0 0) 00010000:00010000:3.0:1785220581.838434:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7249:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.838438:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838457:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000072346d69/0xf62499ab9278b5a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3ea expref: 286 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838468:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000072346d69/0xf62499ab9278b5a2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3ea expref: 286 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.838480:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510685440:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.838510:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.838538:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510683904:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.838554:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.838559:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510683392:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.838573:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838579:0:14166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7566df1/0xf62499ab9278e169 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e82d expref: 248 pid: 13818 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.838591:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510683264:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.838593:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510683648:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.838601:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838603:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838611:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838618:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.838620:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510683776:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.838627:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.838630:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.838636:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.838646:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:3.0:1785220581.838648:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.838654:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a86680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510685312:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.838724:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.838740:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1233:3653:x1871938510682752:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7080us (10093us total) trans 4295070606 rc 0/0 00000100:00100000:3.0:1785220581.838747:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284401 00000100:00100000:1.0:1785220581.838748:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.838782:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.838806:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.838917:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510685184:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.838944:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.838971:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbf100/2, rc 0, transno 4295070021 00000100:00100000:1.0:1785220581.838980:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510681856:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.838993:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99064a80/2, rc 0, transno 4295070018 00000100:00100000:1.0:1785220581.838996:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99064a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510681216:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.839104:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950080 00000100:00100000:2.0:1785220581.839109:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950080 00010000:00010000:2.0:1785220581.839130:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b0fafecc/0xf62499ab9278e249 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8ab expref: 248 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839158:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950336 00000100:00100000:2.0:1785220581.839161:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950336 00010000:00010000:2.0:1785220581.839174:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ae95739e/0xf62499ab9278b444 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d31f expref: 287 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839190:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519949952 00000100:00100000:2.0:1785220581.839193:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519949952 00010000:00010000:2.0:1785220581.839204:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000757bdf8d/0xf62499ab9278b1ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1f2 expref: 288 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839222:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950208 00000100:00100000:2.0:1785220581.839224:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950208 00010000:00010000:2.0:1785220581.839235:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006a19e89c/0xf62499ab9278b364 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2b6 expref: 289 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839254:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100334 00000100:00100000:2.0:1785220581.839261:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+289:2407:x1871938519949312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.839269:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.839274:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd263614/0xf62499ab9278b00e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d112 expref: 289 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839283:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd263614/0xf62499ab9278b00e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d112 expref: 289 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839298:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd263614/0xf62499ab9278b00e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d112 expref: 289 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839308:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7243:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.839311:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7243:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.839316:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839323:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839343:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.839368:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+288:2407:x1871938519949312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (5574us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.839375:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100334 00000100:00100000:2.0:1785220581.839391:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100335 00000100:00100000:2.0:1785220581.839398:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a84000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+248:2406:x1871938519950080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.839406:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.839410:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b0fafecc/0xf62499ab9278e249 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8ab expref: 248 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839420:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b0fafecc/0xf62499ab9278e249 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8ab expref: 248 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.839422:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510685184 00000100:00100000:3.0:1785220581.839426:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510685184 00010000:00010000:2.0:1785220581.839433:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0fafecc/0xf62499ab9278e249 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8ab expref: 248 pid: 14206 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.839448:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284411 00010000:00010000:2.0:1785220581.839449:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7274:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.839452:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7274:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.839454:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1233:3655:x1871938510685184:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.839456:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839463:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.839468:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29305 count 1 00010000:00010000:2.0:1785220581.839477:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.839480:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839492:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000001196ed2/0xf62499ab9278e632 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea64 expref: 247 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839494:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a84000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+247:2406:x1871938519950080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (772us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.839500:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7279:0x0].0x0 since=23s v=(0 0) 00000100:00100000:2.0:1785220581.839501:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100335 00010000:00010000:3.0:1785220581.839503:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7279:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.839507:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839513:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100336 00000100:00100000:2.0:1785220581.839518:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a86a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+287:2406:x1871938519950336:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.839525:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000001196ed2/0xf62499ab9278e632 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea64 expref: 246 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839525:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.839529:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ae95739e/0xf62499ab9278b444 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d31f expref: 287 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839539:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ae95739e/0xf62499ab9278b444 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d31f expref: 287 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839541:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000001196ed2/0xf62499ab9278e632 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea64 expref: 246 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.839552:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510685568:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.839552:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ae95739e/0xf62499ab9278b444 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d31f expref: 287 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839566:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7247:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.839569:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7247:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.839573:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839580:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.839581:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.839596:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.839600:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839608:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839613:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a86a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+286:2406:x1871938519950336:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (864us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.839617:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839619:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100336 00010000:00010000:3.0:1785220581.839625:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.839631:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100337 00010000:00010000:3.0:1785220581.839633:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.839635:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+285:2406:x1871938519949952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.839642:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.839646:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000757bdf8d/0xf62499ab9278b1ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1f2 expref: 285 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839655:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000757bdf8d/0xf62499ab9278b1ab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1f2 expref: 285 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839666:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000757bdf8d/0xf62499ab9278b1ab lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d1f2 expref: 285 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839679:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7245:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.839683:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7245:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.839686:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839693:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.839709:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.839740:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1170:3653:x1871938510683520:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7250us (7571us total) trans 4295070022 rc 0/0 00000100:00100000:3.0:1785220581.839746:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284405 00000100:00100000:3.0:1785220581.839779:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100338 00000100:00100000:3.0:1785220581.839787:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a84e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+284:2406:x1871938519950208:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.839793:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+284:2406:x1871938519949952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 87us (941us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.839798:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.839803:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006a19e89c/0xf62499ab9278b364 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2b6 expref: 284 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839812:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006a19e89c/0xf62499ab9278b364 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2b6 expref: 284 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839825:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006a19e89c/0xf62499ab9278b364 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2b6 expref: 284 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839890:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7246:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.839893:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7246:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.839898:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839905:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.839926:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.839945:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a84e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+283:2406:x1871938519950208:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 160us (1138us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.839993:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840006:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840019:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840026:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840034:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.840122:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1170:3655:x1871938510681984:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18587us (18679us total) trans 4295070023 rc 0/0 00000100:00100000:0.0:1785220581.840128:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284398 00010000:00010000:0.0:1785220581.840149:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840158:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840166:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840172:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840181:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.840245:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1170:3653:x1871938510683136:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7877us (10425us total) trans 4295070024 rc 0/0 00000100:00100000:0.0:1785220581.840255:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284404 00010000:00010000:0.0:1785220581.840280:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840287:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840295:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840301:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840310:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:0.0:1785220581.840379:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+1234:3655:x1871938510682880:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8152us (11577us total) trans 4295070607 rc 0/0 00000100:00100000:0.0:1785220581.840384:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284403 00010000:00010000:0.0:1785220581.840405:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:0.0:1785220581.840410:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840418:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.840421:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840430:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840442:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840449:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a4f7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840464:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220581.840467:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840472:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.840474:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840481:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840489:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840495:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a4f99 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840504:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.840507:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840513:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.840516:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840522:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840529:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840535:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a4f68 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7243:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840545:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.840548:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840553:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.840555:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840561:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840569:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840576:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a4f92 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840586:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.840588:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840593:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.840596:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840603:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840610:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840616:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a4f8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.840625:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.840626:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.840628:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.840645:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.840677:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601c000/2, rc 0, transno 4295070022 00000100:00100000:0.0:1785220581.840681:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510683520:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.840688:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10a300/2, rc 0, transno 4295070024 00000100:00100000:0.0:1785220581.840690:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510683136:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.840696:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c0000/2, rc 0, transno 4295070606 00000100:00100000:0.0:1785220581.840702:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510682752:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.840744:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510685696:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.840777:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.841008:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c0e00/2, rc 0, transno 4295070607 00000100:00100000:1.0:1785220581.841018:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510682880:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.841027:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbc000/2, rc 0, transno 4295070023 00000100:00100000:1.0:1785220581.841030:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510681984:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.841419:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100337 00000100:00100000:2.0:1785220581.841475:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510685312 00000100:00100000:2.0:1785220581.841481:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510685312 00000100:00100000:2.0:1785220581.841548:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510685696 00000100:00100000:2.0:1785220581.841552:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510685696 00000100:00100000:1.0:1785220581.841582:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284412 00000100:00100000:1.0:1785220581.841591:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1235:3653:x1871938510685312:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.841612:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29306 count 1 00010000:00010000:1.0:1785220581.841632:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.841650:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001315741b/0xf62499ab9278e704 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eae2 expref: 246 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.841657:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x727a:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220581.841661:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.841665:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 4/0,1 mode: --/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.841690:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001315741b/0xf62499ab9278e704 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eae2 expref: 246 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.841702:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001315741b/0xf62499ab9278e704 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eae2 expref: 246 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.841715:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510685824:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.841783:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.842069:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284413 00000100:00100000:2.0:1785220581.842078:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1235:3653:x1871938510685696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.842120:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29307 count 1 00010000:00010000:2.0:1785220581.842143:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842155:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fbd975b3/0xf62499ab9278e838 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb44 expref: 246 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842162:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x727b:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220581.842165:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.842169:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842189:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fbd975b3/0xf62499ab9278e838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb44 expref: 246 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842197:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fbd975b3/0xf62499ab9278e838 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb44 expref: 246 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.842206:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510685952:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.842237:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.842255:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842263:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842272:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842279:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.842298:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.842572:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1170:3655:x1871938510683008:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10906us (13833us total) trans 4295070025 rc 0/0 00000100:00100000:2.0:1785220581.842581:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284402 00000100:00100000:3.0:1785220581.842598:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100338 00010000:00010000:0.0:1785220581.842660:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.842666:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.842675:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.842680:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.842688:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.842691:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510684416:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.842704:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.842709:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.842712:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a4f84 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7245:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.842731:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.842733:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.842736:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.842752:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb1c00/2, rc 0, transno 4295070025 00000100:00100000:1.0:1785220581.842772:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510683008:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.842783:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510686208:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.842808:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.842834:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510686208 00000100:00100000:1.0:1785220581.842838:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510686208 00000100:00100000:1.0:1785220581.842869:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284414 00000100:00100000:1.0:1785220581.842876:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1236:3655:x1871938510686208:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.842890:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29308 count 1 00010000:00010000:1.0:1785220581.842902:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.842912:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000079766ded/0xf62499ab9278e926 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebad expref: 246 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.842919:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x727c:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220581.842922:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.842926:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.842945:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000079766ded/0xf62499ab9278e926 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebad expref: 246 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.842955:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000079766ded/0xf62499ab9278e926 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebad expref: 246 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.842964:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510686976:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.842997:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.842999:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510686464:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.843032:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.843037:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510686336:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.843050:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510686464 00000100:00100000:1.0:1785220581.843053:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510686464 00000100:00100000:0.0:1785220581.843070:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510686080:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.843074:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284415 00000100:00100000:1.0:1785220581.843079:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1170:3653:x1871938510686464:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.843110:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29260 count 1 00010000:00010000:3.0:1785220581.843174:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.843180:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.843186:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.843186:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.843198:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009629e7c5/0xf62499ab9278b857 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d548 expref: 281 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.843206:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x724c:0x0].0x0 since=25s v=(0 0) 00010000:00010000:3.0:1785220581.843207:0:13836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000296f95c9/0xf62499ab9278dede lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e71c expref: 246 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.843209:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.843213:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.843219:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510682496:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.843230:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.843232:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009629e7c5/0xf62499ab9278b857 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d548 expref: 281 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.843239:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.843243:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009629e7c5/0xf62499ab9278b857 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d548 expref: 281 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.843246:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.843254:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510687104:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.843255:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.843263:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.843284:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.843429:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f900 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.843466:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.843486:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510686848:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.843506:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.843535:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510686720:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.843557:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.843701:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510686336 00000100:00100000:2.0:1785220581.843705:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510686336 00000100:00100000:2.0:1785220581.843730:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510686848 00000100:00100000:2.0:1785220581.843733:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510686848 00000100:00100000:2.0:1785220581.843749:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510686720 00000100:00100000:2.0:1785220581.843751:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510686720 00000100:00100000:2.0:1785220581.843778:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284416 00000100:00100000:2.0:1785220581.843785:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1173:3652:x1871938510686336:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.843799:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29259 count 1 00010000:00010000:2.0:1785220581.843811:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.843823:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b123fc71/0xf62499ab9278b75b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4bc expref: 281 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.843830:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x724b:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.843833:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.843837:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.843856:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b123fc71/0xf62499ab9278b75b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4bc expref: 281 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.843866:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b123fc71/0xf62499ab9278b75b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4bc expref: 281 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.843882:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510687232:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.843910:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.843926:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284417 00000100:00100000:2.0:1785220581.843933:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1173:3652:x1871938510686848:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.843945:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29263 count 1 00010000:00010000:2.0:1785220581.843957:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.843967:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000603e267f/0xf62499ab9278bb3d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d698 expref: 281 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.843974:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x724f:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.843977:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.843980:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.843998:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000603e267f/0xf62499ab9278bb3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d698 expref: 281 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.844006:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000603e267f/0xf62499ab9278bb3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d698 expref: 281 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.844015:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510687360:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.844038:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df010c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.844054:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284418 00000100:00100000:2.0:1785220581.844061:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1173:3655:x1871938510686720:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.844071:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29262 count 1 00010000:00010000:2.0:1785220581.844081:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.844114:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ddb3b86e/0xf62499ab9278b9c3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d60c expref: 281 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.844121:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x724e:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.844123:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.844126:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.844157:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ddb3b86e/0xf62499ab9278b9c3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d60c expref: 281 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.844165:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ddb3b86e/0xf62499ab9278b9c3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d60c expref: 281 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.844174:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510687488:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.844195:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.845162:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.845172:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.845182:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.845186:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.845196:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.845213:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.845221:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a4f61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.845243:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.845245:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.845246:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.847285:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.847321:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1237:3653:x1871938510681728:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27533us (27597us total) trans 4295070608 rc 0/0 00000100:00100000:0.0:1785220581.847330:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510686080 00000100:00100000:3.0:1785220581.847332:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284397 00000100:00100000:0.0:1785220581.847337:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510686080 00000100:00100000:0.0:1785220581.847375:0:14169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284419 00000100:00100000:0.0:1785220581.847384:0:14169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1174:3654:x1871938510686080:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.847415:0:14169:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29258 count 1 00010000:00010000:0.0:1785220581.847440:0:14169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.847453:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.847459:0:14169:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b1d51a8d/0xf62499ab9278b658 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d45a expref: 281 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.847467:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x724a:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.847472:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.847477:0:14169:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.847479:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.847510:0:14169:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b1d51a8d/0xf62499ab9278b658 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d45a expref: 281 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.847524:0:14169:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b1d51a8d/0xf62499ab9278b658 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d45a expref: 281 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.847531:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.847545:0:14169:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510687616:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.847546:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.847570:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.847640:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510684544:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.847662:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.847689:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510685824:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.847702:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.847719:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950464 00000100:00100000:1.0:1785220581.847722:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950464 00010000:00010000:1.0:1785220581.847743:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003dfe8ac8/0xf62499ab9278e528 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea02 expref: 246 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.847775:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950592 00000100:00100000:3.0:1785220581.847777:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510684672:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.847778:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950592 00010000:00010000:1.0:1785220581.847790:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e3aebb89/0xf62499ab9278e361 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e91b expref: 247 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.847792:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.847806:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950720 00000100:00100000:1.0:1785220581.847808:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950720 00010000:00010000:1.0:1785220581.847819:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d039a7a4/0xf62499ab9278b524 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d39d expref: 282 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.847820:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510685440:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.847835:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950976 00000100:00100000:1.0:1785220581.847838:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950976 00010000:00010000:3.0:1785220581.847846:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.847849:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000001196ed2/0xf62499ab9278e632 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea64 expref: 248 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.847870:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510685056:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.847871:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519950848 00000100:00100000:1.0:1785220581.847873:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519950848 00010000:00010000:3.0:1785220581.847879:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.847884:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000072346d69/0xf62499ab9278b5a2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3ea expref: 283 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.847897:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100339 00000100:00100000:1.0:1785220581.847902:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100340 00000100:00100000:3.0:1785220581.847904:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+248:2407:x1871938519950464:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220581.847907:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+248:2407:x1871938519950592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.847914:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.847915:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.847920:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e3aebb89/0xf62499ab9278e361 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e91b expref: 248 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.847921:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003dfe8ac8/0xf62499ab9278e528 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea02 expref: 248 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.847933:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003dfe8ac8/0xf62499ab9278e528 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea02 expref: 248 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.847933:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e3aebb89/0xf62499ab9278e361 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e91b expref: 248 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.847954:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dfe8ac8/0xf62499ab9278e528 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea02 expref: 248 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.847954:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e3aebb89/0xf62499ab9278e361 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e91b expref: 248 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.847968:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7278:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.847968:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7275:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.847972:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7278:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.847972:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7275:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.847977:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.847977:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.847987:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.847987:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.848006:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.848006:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.848068:0:14169:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.848126:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100341 00000100:00100000:0.0:1785220581.848135:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+283:2407:x1871938519950720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.848164:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.848170:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d039a7a4/0xf62499ab9278b524 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d39d expref: 283 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848180:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d039a7a4/0xf62499ab9278b524 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d39d expref: 283 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848194:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d039a7a4/0xf62499ab9278b524 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d39d expref: 283 pid: 8440 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848204:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7248:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.848206:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7248:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.848210:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848218:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.848229:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.848234:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.848245:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519951104 00000100:00100000:2.0:1785220581.848248:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519951104 00000100:00100000:0.0:1785220581.848256:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+282:2407:x1871938519950720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 124us (724us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.848263:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100341 00010000:00010000:2.0:1785220581.848266:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001315741b/0xf62499ab9278e704 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eae2 expref: 247 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.848278:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100342 00000100:00100000:2.0:1785220581.848283:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100343 00000100:00100000:0.0:1785220581.848285:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+247:2407:x1871938519950976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.848289:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+281:2406:x1871938519950848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.848292:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.848295:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.848297:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000001196ed2/0xf62499ab9278e632 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea64 expref: 247 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848299:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000072346d69/0xf62499ab9278b5a2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3ea expref: 281 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848307:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000001196ed2/0xf62499ab9278e632 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea64 expref: 247 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848309:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000072346d69/0xf62499ab9278b5a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3ea expref: 281 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848319:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7279:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.848321:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000072346d69/0xf62499ab9278b5a2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3ea expref: 281 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848322:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7279:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.848326:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848331:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7249:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.848334:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7249:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.848334:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848338:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848345:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848345:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.848360:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.848362:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+247:2407:x1871938519950976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (815us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.848368:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100342 00000100:00100000:2.0:1785220581.848374:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+280:2406:x1871938519950848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (804us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.848380:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100343 00000100:00100000:0.0:1785220581.848388:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100344 00000100:00100000:0.0:1785220581.848394:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c304450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+246:2406:x1871938519951104:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.848400:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848401:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.848406:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001315741b/0xf62499ab9278e704 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eae2 expref: 246 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848409:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848415:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001315741b/0xf62499ab9278e704 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eae2 expref: 246 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848418:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848426:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848428:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001315741b/0xf62499ab9278e704 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eae2 expref: 246 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848437:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.848440:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.848442:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.848444:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848451:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.848465:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.848481:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c304450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+245:2406:x1871938519951104:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.848487:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100344 00000100:00100000:0.0:1785220581.848526:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510686592:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.848558:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.848582:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1175:3654:x1871938510684928:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10844us (11016us total) trans 4295070026 rc 0/0 00000004:00080000:0.0:1785220581.848585:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94911180/2, rc 0, transno 4295070026 00000100:00100000:2.0:1785220581.848589:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284410 00000100:00100000:0.0:1785220581.848594:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510684928:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.848604:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510686592 00000100:00100000:2.0:1785220581.848607:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510686592 00000100:00100000:2.0:1785220581.848628:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284420 00000100:00100000:2.0:1785220581.848633:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b0e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1175:3654:x1871938510686592:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.848646:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29261 count 1 00010000:00010000:2.0:1785220581.848661:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848673:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e28450ee/0xf62499ab9278b8f8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5a3 expref: 279 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848680:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x724d:0x0].0x0 since=25s v=(0 0) 00010000:00010000:2.0:1785220581.848682:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.848685:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848704:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e28450ee/0xf62499ab9278b8f8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5a3 expref: 279 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848713:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e28450ee/0xf62499ab9278b8f8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5a3 expref: 279 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.848722:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1edc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510687872:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.848745:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.848769:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.848775:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848783:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.848787:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848802:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848813:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848820:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a4fbc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7249:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.848992:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.848994:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.848995:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.849029:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.849082:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519951232 00000100:00100000:0.0:1785220581.849120:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519951232 00010000:00010000:0.0:1785220581.849138:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009cabec6c/0xf62499ab9278e480 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e984 expref: 245 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.849161:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100345 00000100:00100000:0.0:1785220581.849168:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+245:2406:x1871938519951232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.849177:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.849182:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009cabec6c/0xf62499ab9278e480 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e984 expref: 245 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.849191:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009cabec6c/0xf62499ab9278e480 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e984 expref: 245 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.849204:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cabec6c/0xf62499ab9278e480 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e984 expref: 245 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.849213:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7277:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.849216:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7277:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.849220:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.849229:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.849258:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.849282:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+244:2406:x1871938519951232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.849288:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100345 00010000:00010000:2.0:1785220581.849424:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.849434:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.849441:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.849450:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.849469:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.850080:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+243:2407:x1871938519950592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2169us (2597us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.850236:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100340 00010000:00010000:1.0:1785220581.850264:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850274:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850281:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850288:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850304:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220581.850321:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1237:3654:x1871938510684160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13400us (14185us total) trans 4295070609 rc 0/0 00000100:00100000:2.0:1785220581.850329:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284408 00010000:00010000:2.0:1785220581.850346:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.850351:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850359:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.850362:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850371:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850384:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.850389:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1237:3653:x1871938510684288:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13965us (14398us total) trans 4295070610 rc 0/0 00010000:00010000:2.0:1785220581.850392:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a4fae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.850395:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284406 00010000:00010000:2.0:1785220581.850410:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.850414:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850415:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850424:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850426:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.850430:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850438:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850439:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850447:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850448:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850455:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a4fa0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850460:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.850480:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.850484:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850490:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.850494:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850501:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850510:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 1/0,0 mode: --/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.850518:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a4fca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.850521:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1237:3653:x1871938510685312:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8932us (9875us total) trans 4295070611 rc 0/0 00000100:00100000:1.0:1785220581.850527:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284412 00010000:00010000:2.0:1785220581.850543:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.850545:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.850546:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.850547:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850555:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850562:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850568:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850581:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.850649:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1237:3654:x1871938510684032:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14035us (14595us total) trans 4295070612 rc 0/0 00000100:00100000:1.0:1785220581.850658:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284407 00010000:00010000:1.0:1785220581.850678:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.850683:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850690:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.850693:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850700:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850710:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850717:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a4fa7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.850730:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.850732:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.850734:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.850739:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.850751:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.850787:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510687744:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.850819:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.850859:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a86680/2, rc 0, transno 4295070611 00000100:00100000:1.0:1785220581.850868:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a86680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510685312:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.850880:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a84380/2, rc 0, transno 4295070610 00000100:00100000:1.0:1785220581.850883:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510684288:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.850893:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ee300/2, rc 0, transno 4295070608 00000100:00100000:1.0:1785220581.850896:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510681728:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.850994:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510687744 00000100:00100000:0.0:1785220581.850999:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510687744 00000100:00100000:0.0:1785220581.851037:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284421 00000100:00100000:0.0:1785220581.851045:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1176:3653:x1871938510687744:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.851068:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29264 count 1 00010000:00010000:0.0:1785220581.851105:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851120:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000820f23ee/0xf62499ab9278bbc2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6de expref: 279 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851129:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7250:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.851133:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7250:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.851137:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851163:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000820f23ee/0xf62499ab9278bbc2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6de expref: 279 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851175:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000820f23ee/0xf62499ab9278bbc2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6de expref: 279 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.851187:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510688000:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.851243:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.851314:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510688384:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.851339:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.851371:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510688384 00000100:00100000:2.0:1785220581.851376:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510688384 00000100:00100000:0.0:1785220581.851378:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510688128:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.851398:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284422 00000100:00100000:0.0:1785220581.851399:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.851405:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1237:3652:x1871938510688384:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.851417:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29311 count 1 00000100:00100000:0.0:1785220581.851418:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510688128 00000100:00100000:0.0:1785220581.851421:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510688128 00010000:00010000:2.0:1785220581.851429:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.851441:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000051c59d5a/0xf62499ab9278ec0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed12 expref: 242 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.851441:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284423 00000100:00100000:0.0:1785220581.851446:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1238:3654:x1871938510688128:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.851448:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x727f:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220581.851451:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.851455:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220581.851456:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29309 count 1 00010000:00010000:0.0:1785220581.851466:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.851474:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000051c59d5a/0xf62499ab9278ec0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed12 expref: 242 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851479:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b3f89ea1/0xf62499ab9278ea30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec2b expref: 242 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851487:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x727d:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220581.851490:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000051c59d5a/0xf62499ab9278ec0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed12 expref: 242 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851490:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.851494:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.851501:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510688512:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.851509:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b3f89ea1/0xf62499ab9278ea30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec2b expref: 242 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.851518:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b3f89ea1/0xf62499ab9278ea30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec2b expref: 242 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.851526:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.851534:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0000_UUID:14145:1871938510688640:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.851555:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.851580:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a85f80/2, rc 0, transno 4295070609 00000100:00100000:0.0:1785220581.851588:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510684160:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.851617:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220581.851624:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510688768:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.851646:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.851684:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb0e00/2, rc 0, transno 4295070612 00000100:00100000:0.0:1785220581.851692:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510684032:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.851720:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510688768 00000100:00100000:0.0:1785220581.851721:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510688896:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.851724:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510688768 00000100:00100000:2.0:1785220581.851746:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284424 00000100:00100000:2.0:1785220581.851751:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1239:3652:x1871938510688768:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.851773:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29312 count 1 00000100:00100000:0.0:1785220581.851773:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.851785:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.851790:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220581.851796:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510688256:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.851797:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b27025b1/0xf62499ab9278ecd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed74 expref: 242 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.851804:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7280:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220581.851808:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7280:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.851811:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.851813:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.851846:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b27025b1/0xf62499ab9278ecd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed74 expref: 242 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.851862:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b27025b1/0xf62499ab9278ecd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed74 expref: 242 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.851871:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510689024:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.851909:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.851922:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510688896 00000100:00100000:2.0:1785220581.851925:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510688896 00000100:00100000:2.0:1785220581.851947:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510688256 00000100:00100000:2.0:1785220581.851949:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510688256 00000100:00100000:2.0:1785220581.851966:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284425 00000100:00100000:2.0:1785220581.851972:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1241:3654:x1871938510688896:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.851984:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29313 count 1 00010000:00010000:2.0:1785220581.851996:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.852007:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b721d242/0xf62499ab9278ede1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edf2 expref: 242 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.852013:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7281:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220581.852016:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7281:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.852019:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.852037:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b721d242/0xf62499ab9278ede1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edf2 expref: 242 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.852050:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b721d242/0xf62499ab9278ede1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edf2 expref: 242 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.852060:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510689152:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.852103:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+242:2407:x1871938519950464:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 4181us (4633us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.852112:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.852113:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100339 00000100:00100000:1.0:1785220581.852661:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284426 00000100:00100000:1.0:1785220581.852671:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1241:3654:x1871938510688256:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.852684:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29310 count 1 00010000:00010000:1.0:1785220581.852697:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.852708:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000052e74373/0xf62499ab9278eaed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec7f expref: 241 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.852715:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x727e:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220581.852717:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.852721:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.852741:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000052e74373/0xf62499ab9278eaed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec7f expref: 241 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.852750:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000052e74373/0xf62499ab9278eaed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec7f expref: 241 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.852779:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510689280:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.852820:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f300 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.853579:0:14166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001196ed2/0xf62499ab9278e632 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea64 expref: 241 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.853594:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510685568:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.853618:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.853628:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.853635:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.853645:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.853653:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.853677:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.853844:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1242:3655:x1871938510685184:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14390us (14899us total) trans 4295070613 rc 0/0 00000100:00100000:0.0:1785220581.853853:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284411 00010000:00010000:1.0:1785220581.853955:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.853962:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.853970:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.853974:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.853983:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.854001:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.854009:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a4fc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.854028:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.854030:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.854032:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.854057:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a84a80/2, rc 0, transno 4295070613 00000100:00100000:1.0:1785220581.854076:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510685184:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.854202:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220581.854213:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510689408:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.854243:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.854302:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510689408 00000100:00100000:1.0:1785220581.854308:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510689408 00000100:00100000:1.0:1785220581.854342:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284427 00000100:00100000:1.0:1785220581.854350:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1242:3654:x1871938510689408:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.854368:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29314 count 1 00010000:00010000:1.0:1785220581.854385:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.854399:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002326fabd/0xf62499ab9278ee82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee31 expref: 240 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.854406:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7282:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220581.854410:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7282:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.854415:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.854437:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002326fabd/0xf62499ab9278ee82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee31 expref: 240 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.854448:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002326fabd/0xf62499ab9278ee82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee31 expref: 240 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.854461:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510689536:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.854494:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.854726:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.854737:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.854745:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.854966:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855043:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.855069:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510685952:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.855114:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.855342:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1177:3653:x1871938510684800:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17846us (17935us total) trans 4295070027 rc 0/0 00000100:00100000:3.0:1785220581.855351:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284409 00000100:00100000:2.0:1785220581.855377:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.855411:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519951360 00000100:00100000:3.0:1785220581.855417:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519951360 00010000:00010000:3.0:1785220581.855437:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fbd975b3/0xf62499ab9278e838 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb44 expref: 241 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.855438:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510687232:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.855453:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.855461:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100346 00000100:00100000:3.0:1785220581.855469:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+241:2406:x1871938519951360:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.855480:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.855482:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.855485:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fbd975b3/0xf62499ab9278e838 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb44 expref: 241 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855496:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fbd975b3/0xf62499ab9278e838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb44 expref: 241 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855515:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbd975b3/0xf62499ab9278e838 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb44 expref: 241 pid: 14249 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.855522:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.855527:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.855531:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.855536:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855545:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855574:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.855604:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+240:2406:x1871938519951360:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 135us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.855611:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100346 00000100:00100000:3.0:1785220581.855624:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519951488 00000100:00100000:3.0:1785220581.855628:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519951488 00010000:00010000:3.0:1785220581.855640:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000079766ded/0xf62499ab9278e926 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebad expref: 240 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.855664:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519951744 00000100:00100000:3.0:1785220581.855667:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519951744 00010000:00010000:3.0:1785220581.855680:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b123fc71/0xf62499ab9278b75b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4bc expref: 280 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.855698:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100347 00000100:00100000:3.0:1785220581.855704:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+240:2405:x1871938519951488:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.855712:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.855717:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000079766ded/0xf62499ab9278e926 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebad expref: 240 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855726:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000079766ded/0xf62499ab9278e926 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebad expref: 240 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855738:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.855741:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.855745:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855754:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855781:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.855798:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+240:2405:x1871938519951488:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.855805:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100347 00000100:00100000:3.0:1785220581.855818:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100348 00000100:00100000:3.0:1785220581.855825:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+280:2404:x1871938519951744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.855833:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.855837:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b123fc71/0xf62499ab9278b75b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4bc expref: 280 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855847:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b123fc71/0xf62499ab9278b75b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4bc expref: 280 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855861:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b123fc71/0xf62499ab9278b75b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d4bc expref: 280 pid: 14194 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855870:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.855874:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.855878:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855885:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.855904:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.855922:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+279:2404:x1871938519951744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.855929:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100348 00010000:00010000:1.0:1785220581.855994:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.856002:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856011:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.856015:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856025:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856038:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856045:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a4fb5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7248:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856063:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.856065:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.856067:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.856459:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99066300/2, rc 0, transno 4295070027 00000100:00100000:0.0:1785220581.856472:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99066300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510684800:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.856530:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.856542:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510689664:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.856577:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.856648:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510687104:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.856664:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.856670:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.856678:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.856686:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856691:0:14158:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079766ded/0xf62499ab9278e926 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebad expref: 239 pid: 14176 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.856695:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.856701:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510686976:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.856715:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.856716:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.856726:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.856730:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856736:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.856739:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856745:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856767:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856774:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a4fd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856788:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.856790:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.856791:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.856797:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856804:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856812:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856818:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856887:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.856897:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.856901:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856908:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.856911:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856918:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856928:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856936:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a4fd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.856949:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.856952:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.856953:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.857038:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1244:3655:x1871938510686208:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14161us (14228us total) trans 4295070615 rc 0/0 00000100:00100000:1.0:1785220581.857046:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284414 00000100:00100000:2.0:1785220581.857112:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1243:3653:x1871938510685696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15035us (16333us total) trans 4295070614 rc 0/0 00000004:00080000:0.0:1785220581.857112:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94911f80/2, rc 0, transno 4295070614 00000100:00100000:2.0:1785220581.857119:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284413 00000100:00100000:0.0:1785220581.857122:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510685696:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.857185:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510689792:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.857211:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.857302:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510689664 00000100:00100000:1.0:1785220581.857308:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510689664 00000100:00100000:1.0:1785220581.857339:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284428 00000100:00100000:1.0:1785220581.857347:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1177:3653:x1871938510689664:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.857368:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29265 count 1 00010000:00010000:1.0:1785220581.857387:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.857403:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000540a42f2/0xf62499ab9278bce8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d778 expref: 278 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.857413:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7251:0x0].0x0 since=25s v=(0 0) 00010000:00010000:1.0:1785220581.857418:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7251:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.857423:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.857448:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000540a42f2/0xf62499ab9278bce8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d778 expref: 278 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.857462:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000540a42f2/0xf62499ab9278bce8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d778 expref: 278 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.857480:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510689920:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.857529:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000004:00080000:1.0:1785220581.857567:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601dc00/2, rc 0, transno 4295070615 00000100:00100000:1.0:1785220581.857580:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510686208:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.857921:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.857931:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510690048:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.857969:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.858170:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858182:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858191:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858200:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858223:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.858344:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1178:3652:x1871938510686336:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14558us (14877us total) trans 4295070028 rc 0/0 00000100:00100000:0.0:1785220581.858351:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284416 00000100:00100000:0.0:1785220581.858368:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510689792 00000100:00100000:0.0:1785220581.858372:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510689792 00000100:00100000:0.0:1785220581.858394:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510690048 00000100:00100000:0.0:1785220581.858397:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510690048 00000100:00100000:0.0:1785220581.858415:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284429 00000100:00100000:0.0:1785220581.858421:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1244:3653:x1871938510689792:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.858436:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29315 count 1 00010000:00010000:0.0:1785220581.858451:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858470:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009c9ed524/0xf62499ab9278ef1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee9a expref: 238 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858478:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7283:0x0].0x0 since=22s v=(0 0) 00010000:00010000:0.0:1785220581.858481:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7283:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.858485:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858510:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009c9ed524/0xf62499ab9278ef1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee9a expref: 238 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.858521:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009c9ed524/0xf62499ab9278ef1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee9a expref: 238 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.858534:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510690176:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.858568:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.858600:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4b800/2, rc 0, transno 4295070028 00000100:00100000:0.0:1785220581.858610:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510686336:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.858691:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.858698:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.858706:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.858710:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.858718:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.858735:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.858742:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a4fe6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.858776:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.858778:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.858780:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.858791:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510690304:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.858817:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.859493:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.859640:0:14169:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510687616:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.859659:0:14169:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.859723:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93667800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510688000:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.859736:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.859806:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0000_UUID:14145:1871938510688640:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.859820:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.860028:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519951616 00000100:00100000:1.0:1785220581.860034:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519951616 00010000:00010000:1.0:1785220581.860060:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009629e7c5/0xf62499ab9278b857 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d548 expref: 279 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.860207:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100349 00000100:00100000:1.0:1785220581.860217:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+279:2407:x1871938519951616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.860228:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.860232:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009629e7c5/0xf62499ab9278b857 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d548 expref: 279 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.860243:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009629e7c5/0xf62499ab9278b857 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d548 expref: 279 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.860259:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009629e7c5/0xf62499ab9278b857 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d548 expref: 279 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.860271:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.860276:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.860281:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.860290:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.860314:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.860348:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+278:2407:x1871938519951616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (857us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.860356:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100349 00000100:00100000:0.0:1785220581.860526:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860698:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860722:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860742:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860774:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860795:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860815:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860893:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.860926:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.860968:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952000 00000100:00100000:1.0:1785220581.860973:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952000 00010000:00010000:1.0:1785220581.860993:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ddb3b86e/0xf62499ab9278b9c3 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d60c expref: 278 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861016:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952256 00000100:00100000:1.0:1785220581.861019:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952256 00010000:00010000:1.0:1785220581.861030:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e28450ee/0xf62499ab9278b8f8 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5a3 expref: 279 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861055:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952512 00000100:00100000:1.0:1785220581.861058:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952512 00010000:00010000:1.0:1785220581.861072:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000051c59d5a/0xf62499ab9278ec0c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed12 expref: 239 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861115:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952768 00000100:00100000:1.0:1785220581.861118:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952768 00010000:00010000:1.0:1785220581.861130:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b27025b1/0xf62499ab9278ecd7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed74 expref: 240 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861147:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519951872 00000100:00100000:1.0:1785220581.861150:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519951872 00010000:00010000:1.0:1785220581.861160:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000603e267f/0xf62499ab9278bb3d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d698 expref: 280 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861172:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952128 00000100:00100000:1.0:1785220581.861175:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952128 00010000:00010000:1.0:1785220581.861184:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b1d51a8d/0xf62499ab9278b658 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d45a expref: 281 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861196:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952384 00000100:00100000:1.0:1785220581.861198:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952384 00010000:00010000:1.0:1785220581.861210:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000820f23ee/0xf62499ab9278bbc2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6de expref: 282 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861222:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952640 00000100:00100000:1.0:1785220581.861224:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952640 00010000:00010000:1.0:1785220581.861235:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b3f89ea1/0xf62499ab9278ea30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec2b expref: 241 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861245:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519952896 00000100:00100000:1.0:1785220581.861247:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519952896 00010000:00010000:1.0:1785220581.861259:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b721d242/0xf62499ab9278ede1 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edf2 expref: 242 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.861273:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100350 00000100:00100000:1.0:1785220581.861280:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+282:2405:x1871938519952000:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.861290:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.861295:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ddb3b86e/0xf62499ab9278b9c3 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d60c expref: 282 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861304:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ddb3b86e/0xf62499ab9278b9c3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d60c expref: 282 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861321:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861324:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861329:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861336:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861350:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.861379:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+282:2405:x1871938519952000:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (854us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.861386:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100350 00000100:00100000:1.0:1785220581.861401:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100351 00000100:00100000:1.0:1785220581.861407:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce7b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+281:2405:x1871938519952256:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.861414:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.861419:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e28450ee/0xf62499ab9278b8f8 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5a3 expref: 281 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861434:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e28450ee/0xf62499ab9278b8f8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5a3 expref: 281 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861446:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861450:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861454:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861461:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861473:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.861490:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce7b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+281:2405:x1871938519952256:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (793us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.861496:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100351 00000100:00100000:1.0:1785220581.861529:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100352 00000100:00100000:1.0:1785220581.861536:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+242:2405:x1871938519952512:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.861543:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.861548:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000051c59d5a/0xf62499ab9278ec0c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed12 expref: 242 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861558:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000051c59d5a/0xf62499ab9278ec0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed12 expref: 242 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861577:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861580:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861584:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861591:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861603:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.861620:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+242:2405:x1871938519952512:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (897us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.861627:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100352 00000100:00100000:1.0:1785220581.861641:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100353 00000100:00100000:1.0:1785220581.861647:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+241:2405:x1871938519952768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.861654:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.861659:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b27025b1/0xf62499ab9278ecd7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed74 expref: 241 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861668:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b27025b1/0xf62499ab9278ecd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed74 expref: 241 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861679:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7280:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861682:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7280:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861686:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861695:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861706:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.861723:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+241:2405:x1871938519952768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (980us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.861729:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100353 00000100:00100000:1.0:1785220581.861743:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100354 00000100:00100000:1.0:1785220581.861750:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+280:2405:x1871938519951872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.861769:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.861774:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000603e267f/0xf62499ab9278bb3d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d698 expref: 280 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861783:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000603e267f/0xf62499ab9278bb3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d698 expref: 280 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861796:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861799:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861803:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861810:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861825:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.861912:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+280:2405:x1871938519951872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 162us (1136us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.861919:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100354 00000100:00100000:1.0:1785220581.861933:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100355 00000100:00100000:1.0:1785220581.861939:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+279:2405:x1871938519952128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.861946:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.861950:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b1d51a8d/0xf62499ab9278b658 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d45a expref: 279 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861960:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b1d51a8d/0xf62499ab9278b658 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d45a expref: 279 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861973:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b1d51a8d/0xf62499ab9278b658 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d45a expref: 279 pid: 13836 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.861986:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861989:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x724a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.861993:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862000:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862021:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.862040:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+278:2405:x1871938519952128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (1243us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.862047:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100355 00000100:00100000:1.0:1785220581.862060:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100356 00000100:00100000:1.0:1785220581.862066:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+277:2405:x1871938519952384:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.862073:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.862077:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000820f23ee/0xf62499ab9278bbc2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6de expref: 277 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862113:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000820f23ee/0xf62499ab9278bbc2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6de expref: 277 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862126:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000820f23ee/0xf62499ab9278bbc2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d6de expref: 277 pid: 14232 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862136:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7250:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.862139:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7250:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.862143:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862150:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862170:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.862187:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+276:2405:x1871938519952384:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 122us (1373us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.862193:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100356 00000100:00100000:1.0:1785220581.862206:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100357 00000100:00100000:1.0:1785220581.862211:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+240:2405:x1871938519952640:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.862218:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.862222:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b3f89ea1/0xf62499ab9278ea30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec2b expref: 240 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862231:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b3f89ea1/0xf62499ab9278ea30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec2b expref: 240 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862244:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b3f89ea1/0xf62499ab9278ea30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec2b expref: 240 pid: 13908 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862253:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.862256:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.862260:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862268:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862288:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.862306:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+239:2405:x1871938519952640:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (1413us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.862312:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100357 00000100:00100000:1.0:1785220581.862325:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100358 00000100:00100000:1.0:1785220581.862332:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e10450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+238:2405:x1871938519952896:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.862339:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.862344:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b721d242/0xf62499ab9278ede1 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edf2 expref: 238 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862354:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b721d242/0xf62499ab9278ede1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edf2 expref: 238 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862366:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7281:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.862369:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7281:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.862373:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862380:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862392:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.862406:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e10450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+238:2405:x1871938519952896:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (1479us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.862415:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100358 00010000:00010000:1.0:1785220581.862462:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862471:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862481:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862489:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862510:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.862677:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1178:3653:x1871938510686464:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19596us (19643us total) trans 4295070029 rc 0/0 00000100:00100000:1.0:1785220581.862685:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284415 00000100:00100000:1.0:1785220581.862699:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510690304 00000100:00100000:1.0:1785220581.862705:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510690304 00000100:00100000:1.0:1785220581.862730:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284430 00000100:00100000:1.0:1785220581.862737:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1244:3655:x1871938510690048:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.862770:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29316 count 1 00010000:00010000:1.0:1785220581.862789:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862803:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007f4b730c/0xf62499ab9278efcb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eee7 expref: 237 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.862810:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7284:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220581.862813:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7284:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.862817:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.862985:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.862994:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863002:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863010:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863028:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.863141:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1245:3654:x1871938510688128:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11693us (11740us total) trans 4295070616 rc 0/0 00000100:00100000:0.0:1785220581.863148:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284423 00000100:00100000:0.0:1785220581.863164:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284431 00000100:00100000:0.0:1785220581.863171:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1178:3655:x1871938510690304:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.863183:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29266 count 1 00010000:00010000:0.0:1785220581.863196:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863208:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000019f16ca8/0xf62499ab9278bddd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7e1 expref: 275 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863215:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7252:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.863218:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7252:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.863222:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863252:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000019f16ca8/0xf62499ab9278bddd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7e1 expref: 275 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863266:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000019f16ca8/0xf62499ab9278bddd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7e1 expref: 275 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.863282:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510690432:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.863332:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.863349:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863358:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863367:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863374:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863387:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.863475:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1179:3653:x1871938510687744:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12431us (12655us total) trans 4295070030 rc 0/0 00000100:00100000:0.0:1785220581.863482:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284421 00010000:00010000:0.0:1785220581.863504:0:14169:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863512:0:14169:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863525:0:14169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863533:0:14169:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863546:0:14169:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.863618:0:14169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1179:3654:x1871938510686080:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16236us (16340us total) trans 4295070031 rc 0/0 00000100:00100000:0.0:1785220581.863625:0:14169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284419 00010000:00010000:0.0:1785220581.863643:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220581.863649:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863656:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.863660:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863669:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863684:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863690:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c158028/0xf62499ab927a4fdf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863709:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.863713:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863718:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.863721:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863728:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863738:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863744:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5017 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863882:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.863887:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863895:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.863898:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863909:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863919:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863927:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5009 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7250:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863940:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.863944:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863950:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ada00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.863953:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863960:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863971:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863977:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a4ffb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.863996:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.863998:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.863999:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.864343:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007f4b730c/0xf62499ab9278efcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eee7 expref: 237 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.864355:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007f4b730c/0xf62499ab9278efcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eee7 expref: 237 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.864368:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510690560:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.864400:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fcc0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.864425:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.864430:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.864435:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.864481:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93664a80/2, rc 0, transno 4295070616 00000100:00100000:1.0:1785220581.864500:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93664a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510688128:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.864514:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601c700/2, rc 0, transno 4295070031 00000100:00100000:1.0:1785220581.864523:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510686080:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.864679:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93666300/2, rc 0, transno 4295070030 00000100:00100000:0.0:1785220581.864684:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93666300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510687744:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.864693:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4a300/2, rc 0, transno 4295070029 00000100:00100000:0.0:1785220581.864696:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510686464:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.864701:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.864711:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510690688:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.864739:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.864777:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510690688 00000100:00100000:0.0:1785220581.864783:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510690688 00000100:00100000:0.0:1785220581.864819:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284432 00000100:00100000:0.0:1785220581.864888:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1245:3653:x1871938510690688:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.864909:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29317 count 1 00010000:00010000:0.0:1785220581.864927:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.864941:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cae6b937/0xf62499ab9278f088 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef42 expref: 237 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.864948:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7285:0x0].0x0 since=22s v=(0 0) 00010000:00010000:0.0:1785220581.864951:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7285:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.864955:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.864980:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cae6b937/0xf62499ab9278f088 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef42 expref: 237 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.864990:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cae6b937/0xf62499ab9278f088 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef42 expref: 237 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.864999:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510690816:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.865024:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.865144:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510690944:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.865167:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.865256:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510691200:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.865282:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.865311:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.865316:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510691072:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.865338:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.865396:0:14186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510690944 00000100:00100000:0.0:1785220581.865400:0:14186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510690944 00000100:00100000:0.0:1785220581.865431:0:14186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510691200 00000100:00100000:0.0:1785220581.865435:0:14186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510691200 00000100:00100000:0.0:1785220581.865456:0:14186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510691072 00000100:00100000:0.0:1785220581.865459:0:14186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510691072 00000100:00100000:0.0:1785220581.865477:0:14186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284433 00000100:00100000:0.0:1785220581.865483:0:14186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1181:3653:x1871938510690944:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.865496:0:14186:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29267 count 1 00010000:00010000:0.0:1785220581.865512:0:14186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865526:0:14186:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000051f57152/0xf62499ab9278bee7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d858 expref: 275 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865539:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7253:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.865542:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7253:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.865546:0:14186:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865569:0:14186:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000051f57152/0xf62499ab9278bee7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d858 expref: 275 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865580:0:14186:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000051f57152/0xf62499ab9278bee7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d858 expref: 275 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.865591:0:14186:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510691328:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.865622:0:14186:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.865639:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284434 00000100:00100000:0.0:1785220581.865646:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1181:3655:x1871938510691200:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.865657:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29269 count 1 00010000:00010000:0.0:1785220581.865670:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865682:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000394b4b2f/0xf62499ab9278c0f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d95b expref: 275 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865688:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7255:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.865691:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7255:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.865696:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865715:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000394b4b2f/0xf62499ab9278c0f4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d95b expref: 275 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.865724:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000394b4b2f/0xf62499ab9278c0f4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d95b expref: 275 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.865734:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510691456:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.865899:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284435 00000100:00100000:1.0:1785220581.865909:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1181:3655:x1871938510691072:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.865922:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29268 count 1 00010000:00010000:1.0:1785220581.865934:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.865946:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000050d15215/0xf62499ab9278bfa4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8cf expref: 275 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.865952:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7254:0x0].0x0 since=25s v=(0 0) 00010000:00010000:1.0:1785220581.865955:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7254:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.865958:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.865995:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000050d15215/0xf62499ab9278bfa4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8cf expref: 275 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.866003:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000050d15215/0xf62499ab9278bfa4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8cf expref: 275 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.866013:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98404000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510691584:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.866036:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.866589:0:14194:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000603e267f/0xf62499ab9278bb3d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d698 expref: 275 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.866605:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510687360:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.866622:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.866625:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866634:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866641:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866650:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866657:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866681:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.866687:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.866693:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.866700:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.866703:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.866711:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.866728:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.866735:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a4fed lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.866751:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.866752:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.866754:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.866812:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.866854:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1182:3652:x1871938510686848:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22920us (23346us total) trans 4295070032 rc 0/0 00000100:00100000:2.0:1785220581.866862:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284417 00010000:00010000:2.0:1785220581.866898:0:14165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ddb3b86e/0xf62499ab9278b9c3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d60c expref: 274 pid: 14245 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.866909:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510687488:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.866912:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510689536:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.866920:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.866923:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866928:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866935:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866942:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.866949:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.866950:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4ad80/2, rc 0, transno 4295070032 00010000:00010000:2.0:1785220581.866964:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.866984:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953024 00000100:00100000:0.0:1785220581.866989:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953024 00010000:00010000:0.0:1785220581.867014:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002326fabd/0xf62499ab9278ee82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee31 expref: 238 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.867036:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1182:3655:x1871938510686720:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22975us (23478us total) trans 4295070033 rc 0/0 00000100:00100000:2.0:1785220581.867042:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284418 00000100:00100000:0.0:1785220581.867047:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100359 00000100:00100000:0.0:1785220581.867054:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+238:2407:x1871938519953024:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.867064:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.867069:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002326fabd/0xf62499ab9278ee82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee31 expref: 238 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867070:0:14259:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c59d5a/0xf62499ab9278ec0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed12 expref: 238 pid: 14223 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.867082:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510688512:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.867082:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002326fabd/0xf62499ab9278ee82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee31 expref: 237 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867112:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867121:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867139:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867149:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867157:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867169:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.867173:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510686848:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.867183:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.867187:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867194:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.867197:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867204:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867216:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867223:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a4ff4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867240:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.867243:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867250:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.867253:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.867257:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1246:3652:x1871938510688384:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15852us (15917us total) trans 4295070617 rc 0/0 00010000:00010000:1.0:1785220581.867262:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.867264:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284422 00010000:00010000:1.0:1785220581.867272:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867279:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5010 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867291:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.867292:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.867293:0:14198:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b721d242/0xf62499ab9278ede1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edf2 expref: 237 pid: 14170 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867294:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.867299:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.867304:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510689152:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.867314:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867323:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867329:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867338:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.867338:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510689280:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.867345:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867350:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867357:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.867361:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.867365:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867372:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.867375:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867387:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867397:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867404:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5025 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867419:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.867420:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.867422:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.867426:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1246:3654:x1871938510688896:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15453us (15652us total) trans 4295070618 rc 0/0 00000100:00100000:2.0:1785220581.867432:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284425 00000100:00100000:1.0:1785220581.867457:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.867460:0:14232:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b27025b1/0xf62499ab9278ecd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed74 expref: 236 pid: 14196 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.867471:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953152 00000100:00100000:1.0:1785220581.867474:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953152 00000100:00100000:2.0:1785220581.867476:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510689024:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.867486:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867488:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000052e74373/0xf62499ab9278eaed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec7f expref: 236 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867495:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867501:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867510:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.867516:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100360 00010000:00010000:2.0:1785220581.867518:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.867522:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98406a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+236:2407:x1871938519953152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.867531:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.867531:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.867535:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000052e74373/0xf62499ab9278eaed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec7f expref: 236 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867544:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000052e74373/0xf62499ab9278eaed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec7f expref: 236 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867559:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052e74373/0xf62499ab9278eaed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec7f expref: 236 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867570:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.867573:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x727e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.867579:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867587:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.867594:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1246:3652:x1871938510688768:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15842us (15947us total) trans 4295070619 rc 0/0 00000100:00100000:2.0:1785220581.867600:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284424 00010000:00010000:1.0:1785220581.867605:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.867626:0:13867:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e28450ee/0xf62499ab9278b8f8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d5a3 expref: 273 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.867627:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98406a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+235:2407:x1871938519953152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (171us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.867634:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100360 00000100:00100000:2.0:1785220581.867637:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1edc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510687872:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.867647:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867651:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867655:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867659:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867663:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867667:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867671:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867673:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.867680:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.867685:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.867690:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.867752:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b0e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1182:3654:x1871938510686592:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19118us (19193us total) trans 4295070034 rc 0/0 00000100:00100000:1.0:1785220581.867752:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1246:3654:x1871938510688256:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15083us (15937us total) trans 4295070620 rc 0/0 00000100:00100000:2.0:1785220581.867773:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284420 00000100:00100000:2.0:1785220581.867883:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.867914:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002326fabd/0xf62499ab9278ee82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee31 expref: 234 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.867924:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7282:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.867927:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7282:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.867931:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.867942:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.867961:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.867968:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510690176:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.867980:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.867994:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.867998:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868004:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.868006:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868014:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868023:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868029:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a501e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868047:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220581.868050:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868056:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.868060:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868067:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868076:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868082:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006135719f/0xf62499ab927a502c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868257:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.868263:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868271:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.868275:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868282:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.868287:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.868293:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868300:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5002 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x724d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.868307:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510689920:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.868313:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.868314:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.868316:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.868319:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.868333:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb22a00/2, rc 0, transno 4295070619 00000100:00100000:2.0:1785220581.868344:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510688768:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.868353:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991b100/2, rc 0, transno 4295070617 00000100:00100000:2.0:1785220581.868357:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510688384:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.868380:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b48700/2, rc 0, transno 4295070033 00000100:00100000:2.0:1785220581.868384:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510686720:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.868386:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.868388:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.868390:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.868398:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953280 00000004:00080000:2.0:1785220581.868401:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991a680/2, rc 0, transno 4295070620 00000100:00100000:0.0:1785220581.868402:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953280 00000100:00100000:2.0:1785220581.868404:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510688256:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.868411:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fed80/2, rc 0, transno 4295070618 00000100:00100000:2.0:1785220581.868414:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510688896:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.868420:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000540a42f2/0xf62499ab9278bce8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d778 expref: 273 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.868424:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b48a80/2, rc 0, transno 4295070034 00000100:00100000:2.0:1785220581.868427:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510686592:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.868441:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953408 00000100:00100000:0.0:1785220581.868444:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953408 00010000:00010000:0.0:1785220581.868457:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009c9ed524/0xf62499ab9278ef1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee9a expref: 234 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.868487:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100361 00000100:00100000:0.0:1785220581.868494:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+273:2407:x1871938519953280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.868502:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.868507:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000540a42f2/0xf62499ab9278bce8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d778 expref: 273 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868516:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000540a42f2/0xf62499ab9278bce8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d778 expref: 273 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868530:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000540a42f2/0xf62499ab9278bce8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d778 expref: 273 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868540:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7251:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.868543:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7251:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.868546:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868553:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.868609:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.868633:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100362 00000100:00100000:0.0:1785220581.868634:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2407:x1871938519953280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 142us (752us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.868640:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+234:2406:x1871938519953408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.868642:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100361 00010000:00010000:2.0:1785220581.868648:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.868653:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009c9ed524/0xf62499ab9278ef1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee9a expref: 234 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868663:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009c9ed524/0xf62499ab9278ef1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee9a expref: 234 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868674:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c9ed524/0xf62499ab9278ef1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee9a expref: 234 pid: 14250 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868684:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7283:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.868687:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7283:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.868691:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868700:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868716:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.868734:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+233:2406:x1871938519953408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (771us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.868744:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100362 00010000:00010000:0.0:1785220581.868769:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.868775:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868782:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.868787:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+232:2407:x1871938519953024:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1734us (1976us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.868792:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.868793:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100359 00010000:00010000:2.0:1785220581.868799:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.868812:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.868817:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510691840:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.868913:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.868942:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.868949:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510691712:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.868968:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1247:3653:x1871938510689792:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10547us (11756us total) trans 4295070621 rc 0/0 00000100:00100000:0.0:1785220581.868968:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.868991:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510691840 00000100:00100000:0.0:1785220581.868995:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510691840 00000100:00100000:1.0:1785220581.869044:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284426 00000100:00100000:1.0:1785220581.869059:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510691712 00000100:00100000:1.0:1785220581.869063:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510691712 00000100:00100000:1.0:1785220581.869099:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284436 00000100:00100000:1.0:1785220581.869106:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1248:3653:x1871938510691712:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.869125:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29318 count 1 00010000:00010000:1.0:1785220581.869142:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869156:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000be21d2a1/0xf62499ab9278f192 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef96 expref: 231 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869163:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7286:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220581.869166:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7286:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.869170:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869190:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000be21d2a1/0xf62499ab9278f192 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef96 expref: 231 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869202:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000be21d2a1/0xf62499ab9278f192 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef96 expref: 231 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.869220:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98404e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510692224:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.869252:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.869270:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.869274:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869282:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.869285:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869293:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869306:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869313:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a5041 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869327:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.869328:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.869330:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.869337:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869350:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869358:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869365:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869376:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.869444:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1249:3654:x1871938510689408:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15094us (15200us total) trans 4295070622 rc 0/0 00000100:00100000:1.0:1785220581.869450:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284427 00010000:00010000:1.0:1785220581.869465:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.869469:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869476:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.869479:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869487:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869497:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869505:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000653106ba/0xf62499ab927a5033 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869517:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.869519:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.869520:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.869594:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.869603:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.869611:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.869618:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.869637:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.869714:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1182:3653:x1871938510689664:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12368us (13136us total) trans 4295070035 rc 0/0 00000100:00100000:3.0:1785220581.869723:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284428 00010000:00010000:1.0:1785220581.869772:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.869777:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869785:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.869788:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869796:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869808:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869815:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e899db7/0xf62499ab927a503a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7251:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.869838:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.869840:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.869842:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.870143:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510692096:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.870172:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.870193:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1785220581.870198:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510691968:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.870213:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.870273:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510692096 00000100:00100000:1.0:1785220581.870278:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510692096 00000100:00100000:1.0:1785220581.870299:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510691968 00000100:00100000:1.0:1785220581.870301:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510691968 00000100:00100000:1.0:1785220581.870348:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284437 00000100:00100000:1.0:1785220581.870355:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1250:3655:x1871938510692096:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.870370:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29321 count 1 00010000:00010000:1.0:1785220581.870383:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.870398:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fa1202fb/0xf62499ab9278f272 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f037 expref: 231 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.870406:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7289:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220581.870409:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7289:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.870413:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.870434:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fa1202fb/0xf62499ab9278f272 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f037 expref: 231 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.870445:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fa1202fb/0xf62499ab9278f272 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f037 expref: 231 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.870455:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98405c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510692352:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.870487:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f480 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.870555:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284438 00000100:00100000:3.0:1785220581.870564:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1250:3655:x1871938510691968:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.870574:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29320 count 1 00010000:00010000:3.0:1785220581.870585:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.870596:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003c9d2d36/0xf62499ab9278f21e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eff8 expref: 231 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.870603:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7288:0x0].0x0 since=22s v=(0 0) 00010000:00010000:3.0:1785220581.870606:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7288:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.870609:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.870625:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003c9d2d36/0xf62499ab9278f21e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eff8 expref: 231 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.870633:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003c9d2d36/0xf62499ab9278f21e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eff8 expref: 231 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.870649:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510692480:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.870665:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df180 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.870959:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284429 00000100:00100000:2.0:1785220581.870982:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284439 00000100:00100000:2.0:1785220581.870990:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1249:3653:x1871938510691840:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.871002:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29319 count 1 00010000:00010000:2.0:1785220581.871014:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.871025:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007c0e4ed3/0xf62499ab9278f1d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efb9 expref: 231 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.871037:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7287:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220581.871040:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7287:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.871044:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.871065:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007c0e4ed3/0xf62499ab9278f1d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efb9 expref: 231 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.871074:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007c0e4ed3/0xf62499ab9278f1d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efb9 expref: 231 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.871081:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac8a80/2, rc 0, transno 4295070621 00000100:00100000:2.0:1785220581.871097:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510692608:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.871134:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.871232:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.871535:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510689792:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.871552:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955ff480/2, rc 0, transno 4295070035 00000100:00100000:0.0:1785220581.871560:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510689664:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.871570:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98405880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510692864:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.871616:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.871640:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.871645:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98405500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510692736:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.871662:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.871693:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fc380/2, rc 0, transno 4295070622 00000100:00100000:0.0:1785220581.871697:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510689408:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.871753:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510692864 00000100:00100000:1.0:1785220581.871769:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510692864 00000100:00100000:1.0:1785220581.871797:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510692736 00000100:00100000:1.0:1785220581.871799:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510692736 00000100:00100000:1.0:1785220581.871813:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284440 00000100:00100000:1.0:1785220581.871819:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1183:3653:x1871938510692864:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.871888:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29271 count 1 00010000:00010000:1.0:1785220581.871905:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.871919:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bfdeeeda/0xf62499ab9278c301 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da50 expref: 271 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.871925:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7257:0x0].0x0 since=25s v=(0 0) 00010000:00010000:1.0:1785220581.871928:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7257:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.871932:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.871952:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bfdeeeda/0xf62499ab9278c301 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da50 expref: 271 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.871962:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bfdeeeda/0xf62499ab9278c301 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da50 expref: 271 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.871973:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98407800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510693120:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.872001:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.872017:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284441 00000100:00100000:1.0:1785220581.872024:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1183:3653:x1871938510692736:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.872034:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29270 count 1 00010000:00010000:1.0:1785220581.872045:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.872056:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000161b1c26/0xf62499ab9278c20c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d9d2 expref: 271 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.872063:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7256:0x0].0x0 since=25s v=(0 0) 00010000:00010000:1.0:1785220581.872065:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7256:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.872069:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.872111:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000161b1c26/0xf62499ab9278c20c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d9d2 expref: 271 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.872120:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000161b1c26/0xf62499ab9278c20c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d9d2 expref: 271 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.872130:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98406d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510693248:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.872153:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.872261:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510693504:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.872286:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.872308:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510693504 00000100:00100000:0.0:1785220581.872311:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220581.872312:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510693504 00000100:00100000:0.0:1785220581.872316:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510693632:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.872336:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284442 00000100:00100000:0.0:1785220581.872336:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.872341:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1250:3652:x1871938510693504:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.872353:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29322 count 1 00010000:00010000:2.0:1785220581.872366:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872379:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000734f22cc/0xf62499ab9278f2c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f076 expref: 231 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872390:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x728a:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220581.872393:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.872397:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872416:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000734f22cc/0xf62499ab9278f2c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f076 expref: 231 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872425:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000734f22cc/0xf62499ab9278f2c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f076 expref: 231 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.872435:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510693760:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.872478:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df010c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.872490:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510693632 00000100:00100000:2.0:1785220581.872493:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510693632 00000100:00100000:2.0:1785220581.872515:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284443 00000100:00100000:2.0:1785220581.872521:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1251:3652:x1871938510693632:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.872531:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29323 count 1 00010000:00010000:2.0:1785220581.872543:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872554:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006a43c83b/0xf62499ab9278f2fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ae expref: 231 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872567:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x728b:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220581.872570:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.872573:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872591:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006a43c83b/0xf62499ab9278f2fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ae expref: 231 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.872601:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006a43c83b/0xf62499ab9278f2fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ae expref: 231 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.872611:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510693888:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.872633:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.872685:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98406300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510693376:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.872715:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.872737:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98407100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510692992:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.872786:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.872945:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510693376 00000100:00100000:0.0:1785220581.872949:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510693376 00000100:00100000:0.0:1785220581.873008:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510692992 00000100:00100000:0.0:1785220581.873011:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510692992 00000100:00100000:1.0:1785220581.873041:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284444 00000100:00100000:1.0:1785220581.873049:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1185:3655:x1871938510693376:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.873061:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29273 count 1 00010000:00010000:1.0:1785220581.873073:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.873233:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005aaaa02c/0xf62499ab9278c500 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db22 expref: 271 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.873241:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7259:0x0].0x0 since=25s v=(0 0) 00010000:00010000:1.0:1785220581.873243:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7259:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.873247:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.873267:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005aaaa02c/0xf62499ab9278c500 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db22 expref: 271 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.873276:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005aaaa02c/0xf62499ab9278c500 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db22 expref: 271 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.873287:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98405f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510694016:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.873318:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.873369:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284445 00000100:00100000:0.0:1785220581.873377:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1185:3655:x1871938510692992:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.873387:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29272 count 1 00010000:00010000:0.0:1785220581.873398:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.873407:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000923f53d2/0xf62499ab9278c3ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dab9 expref: 271 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.873413:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7258:0x0].0x0 since=25s v=(0 0) 00010000:00010000:0.0:1785220581.873416:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7258:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.873419:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.873436:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000923f53d2/0xf62499ab9278c3ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dab9 expref: 271 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.873446:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000923f53d2/0xf62499ab9278c3ef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dab9 expref: 271 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.873461:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510694144:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.873484:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9d80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.873980:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.874028:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953536 00000100:00100000:2.0:1785220581.874032:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953536 00000100:00100000:1.0:1785220581.874032:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510690432:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.874052:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000019f16ca8/0xf62499ab9278bddd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7e1 expref: 272 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.874052:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.874114:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100363 00000100:00100000:2.0:1785220581.874122:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2407:x1871938519953536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.874139:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.874145:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000019f16ca8/0xf62499ab9278bddd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7e1 expref: 272 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.874164:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000019f16ca8/0xf62499ab9278bddd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7e1 expref: 272 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.874184:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019f16ca8/0xf62499ab9278bddd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d7e1 expref: 272 pid: 14158 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.874197:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7252:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.874200:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7252:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.874206:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.874214:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.874235:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.874254:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.874263:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.874270:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+271:2407:x1871938519953536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 149us (292us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.874273:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.874277:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100363 00010000:00010000:1.0:1785220581.874280:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.874538:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.874697:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1186:3655:x1871938510690304:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11527us (15879us total) trans 4295070036 rc 0/0 00000100:00100000:1.0:1785220581.874705:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284431 00000004:00080000:0.0:1785220581.874784:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94acaa00/2, rc 0, transno 4295070036 00000100:00100000:0.0:1785220581.874803:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510690304:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.874989:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98404380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510694272:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.875019:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.875107:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510694272 00000100:00100000:1.0:1785220581.875112:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510694272 00000100:00100000:1.0:1785220581.875141:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284446 00000100:00100000:1.0:1785220581.875148:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1186:3654:x1871938510694272:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.875163:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29274 count 1 00010000:00010000:1.0:1785220581.875178:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.875192:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004b3d7b96/0xf62499ab9278c673 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dba0 expref: 270 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.875199:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x725a:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.875202:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.875206:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.875233:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004b3d7b96/0xf62499ab9278c673 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dba0 expref: 270 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.875245:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004b3d7b96/0xf62499ab9278c673 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dba0 expref: 270 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.875256:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98405180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510694400:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.875296:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777600 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.875684:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.875692:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.875702:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.875705:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.875714:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.875730:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.875737:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a504f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7252:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.875895:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.875897:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.875899:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.876694:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510690560:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.876717:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.877145:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510690816:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.877163:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.877826:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98404e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510692224:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.877844:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.879441:0:14186:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510691328:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.879467:0:14186:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.879685:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510691456:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.879700:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.880523:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98404000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510691584:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.880540:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.880956:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0000_UUID:14173:1871938510692480:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.880966:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98405180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510694400:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.880973:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.880981:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.881018:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98405c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510692352:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.881030:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.882910:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510692608:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.882935:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.883684:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510694144:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.883704:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.883739:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.883747:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510693760:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.883772:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.883773:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.883790:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.883821:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.883836:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954944 00000100:00100000:3.0:1785220581.883843:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954944 00000100:00100000:0.0:1785220581.883847:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.883864:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.883878:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.883879:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bfdeeeda/0xf62499ab9278c301 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da50 expref: 271 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.883892:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.883913:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.883918:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953664 00000100:00100000:3.0:1785220581.883921:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953664 00000100:00100000:0.0:1785220581.883926:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953920 00000100:00100000:0.0:1785220581.883928:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953920 00010000:00010000:3.0:1785220581.883936:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007f4b730c/0xf62499ab9278efcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eee7 expref: 233 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.883941:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000be21d2a1/0xf62499ab9278f192 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef96 expref: 233 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.883953:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955328 00000100:00100000:3.0:1785220581.883955:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955328 00000100:00100000:0.0:1785220581.883963:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954176 00000100:00100000:0.0:1785220581.883965:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954176 00010000:00010000:3.0:1785220581.883966:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006a43c83b/0xf62499ab9278f2fe lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ae expref: 234 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.883975:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000394b4b2f/0xf62499ab9278c0f4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d95b expref: 272 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.883979:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954816 00000100:00100000:3.0:1785220581.883981:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954816 00010000:00010000:3.0:1785220581.883990:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007c0e4ed3/0xf62499ab9278f1d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efb9 expref: 235 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.883991:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954432 00000100:00100000:0.0:1785220581.883993:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954432 00010000:00010000:0.0:1785220581.884002:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fa1202fb/0xf62499ab9278f272 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f037 expref: 236 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884005:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955072 00000100:00100000:3.0:1785220581.884007:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955072 00000100:00100000:0.0:1785220581.884012:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954688 00000100:00100000:0.0:1785220581.884014:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954688 00010000:00010000:3.0:1785220581.884016:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000734f22cc/0xf62499ab9278f2c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f076 expref: 237 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884024:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003c9d2d36/0xf62499ab9278f21e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eff8 expref: 238 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884029:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100364 00000100:00100000:0.0:1785220581.884037:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100365 00000100:00100000:3.0:1785220581.884038:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2404:x1871938519954944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220581.884042:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90703450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+238:2407:x1871938519953664:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.884053:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.884053:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.884059:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bfdeeeda/0xf62499ab9278c301 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da50 expref: 272 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884059:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007f4b730c/0xf62499ab9278efcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eee7 expref: 238 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884073:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bfdeeeda/0xf62499ab9278c301 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da50 expref: 272 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884073:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007f4b730c/0xf62499ab9278efcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eee7 expref: 238 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884127:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f4b730c/0xf62499ab9278efcb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eee7 expref: 238 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884144:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7284:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884149:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7284:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884154:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884164:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884186:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.884232:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90703450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+237:2407:x1871938519953664:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 188us (458us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.884239:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100365 00010000:00010000:3.0:1785220581.884244:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7257:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.884247:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7257:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.884251:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884258:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100366 00010000:00010000:3.0:1785220581.884265:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884265:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+236:2407:x1871938519953920:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.884272:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.884276:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000be21d2a1/0xf62499ab9278f192 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef96 expref: 236 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884278:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.884285:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000be21d2a1/0xf62499ab9278f192 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef96 expref: 236 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884296:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+272:2404:x1871938519954944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 260us (558us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.884297:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be21d2a1/0xf62499ab9278f192 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef96 expref: 236 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884302:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100364 00010000:00010000:0.0:1785220581.884311:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7286:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884314:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7286:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884318:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884325:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884338:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.884339:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100367 00000100:00100000:3.0:1785220581.884345:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+235:2404:x1871938519955328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.884353:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220581.884354:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+235:2407:x1871938519953920:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (563us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.884357:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006a43c83b/0xf62499ab9278f2fe lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ae expref: 235 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884360:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100366 00010000:00010000:3.0:1785220581.884367:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006a43c83b/0xf62499ab9278f2fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ae expref: 234 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884371:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100368 00000100:00100000:0.0:1785220581.884377:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+271:2407:x1871938519954176:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.884380:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.884383:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884383:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.884387:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884388:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000394b4b2f/0xf62499ab9278c0f4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d95b expref: 271 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884396:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884398:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000394b4b2f/0xf62499ab9278c0f4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d95b expref: 271 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884410:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000394b4b2f/0xf62499ab9278c0f4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d95b expref: 271 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884412:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.884425:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7255:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.884427:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+234:2404:x1871938519955328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (607us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.884428:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7255:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884432:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884434:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100367 00010000:00010000:0.0:1785220581.884440:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884446:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100369 00000100:00100000:3.0:1785220581.884452:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+233:2404:x1871938519954816:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.884456:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.884459:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.884463:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007c0e4ed3/0xf62499ab9278f1d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efb9 expref: 233 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884471:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+270:2407:x1871938519954176:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (624us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.884473:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007c0e4ed3/0xf62499ab9278f1d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efb9 expref: 233 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884478:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100368 00010000:00010000:3.0:1785220581.884485:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007c0e4ed3/0xf62499ab9278f1d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efb9 expref: 233 pid: 14178 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884489:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100370 00000100:00100000:0.0:1785220581.884495:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+232:2407:x1871938519954432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.884496:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7287:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.884499:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7287:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884501:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.884503:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884505:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fa1202fb/0xf62499ab9278f272 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f037 expref: 232 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884512:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884515:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fa1202fb/0xf62499ab9278f272 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f037 expref: 232 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884527:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa1202fb/0xf62499ab9278f272 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f037 expref: 232 pid: 14201 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884534:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.884541:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7289:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884544:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7289:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884548:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884550:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+231:2404:x1871938519954816:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (686us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220581.884556:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884557:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100369 00000100:00100000:3.0:1785220581.884570:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100371 00010000:00010000:0.0:1785220581.884570:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.884576:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89effb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+230:2404:x1871938519955072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.884583:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220581.884584:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+230:2407:x1871938519954432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (705us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220581.884587:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000734f22cc/0xf62499ab9278f2c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f076 expref: 230 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884590:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100370 00010000:00010000:3.0:1785220581.884596:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000734f22cc/0xf62499ab9278f2c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f076 expref: 229 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.884600:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100372 00000100:00100000:0.0:1785220581.884605:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+229:2407:x1871938519954688:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.884609:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000734f22cc/0xf62499ab9278f2c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f076 expref: 229 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884611:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.884615:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003c9d2d36/0xf62499ab9278f21e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eff8 expref: 229 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884618:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.884621:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884624:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003c9d2d36/0xf62499ab9278f21e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eff8 expref: 228 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884625:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884633:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884637:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c9d2d36/0xf62499ab9278f21e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eff8 expref: 228 pid: 14267 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884652:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7288:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.884654:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.884654:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7288:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.884659:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884666:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.884669:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89effb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+227:2404:x1871938519955072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (777us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.884676:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100371 00010000:00010000:0.0:1785220581.884682:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.884694:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+226:2407:x1871938519954688:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 89us (780us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.884700:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100372 00010000:00010000:3.0:1785220581.884707:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884715:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884722:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884725:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884731:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884734:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884740:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.884745:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.884748:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.884754:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220581.885019:0:14251:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bfdeeeda/0xf62499ab9278c301 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7da50 expref: 269 pid: 14188 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.885032:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98407800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510693120:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.885049:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885057:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885064:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885072:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885079:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885119:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.885215:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1188:3655:x1871938510691200:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19294us (19656us total) trans 4295070037 rc 0/0 00000100:00100000:0.0:1785220581.885224:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284434 00010000:00010000:0.0:1785220581.885248:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.885261:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.885269:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.885276:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.885292:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220581.885548:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1253:3652:x1871938510693504:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13205us (13260us total) trans 4295070623 rc 0/0 00000100:00100000:1.0:1785220581.885554:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1187:3653:x1871938510692864:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13735us (13938us total) trans 4295070038 rc 0/0 00000100:00100000:3.0:1785220581.885557:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284442 00000100:00100000:1.0:1785220581.885560:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284440 00010000:00010000:3.0:1785220581.885582:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.885592:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.885592:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98405f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510694016:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.885605:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885606:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.885613:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.885622:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00010000:00010000:1.0:1785220581.885626:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885634:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885643:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885650:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885658:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:3.0:1785220581.885698:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+1254:3653:x1871938510691840:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14710us (16784us total) trans 4295070625 rc 0/0 00000100:00100000:3.0:1785220581.885704:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284439 00000100:00100000:1.0:1785220581.885730:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1253:3655:x1871938510692096:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15376us (15557us total) trans 4295070626 rc 0/0 00000100:00100000:1.0:1785220581.885736:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284437 00010000:00010000:1.0:1785220581.885771:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:1.0:1785220581.885777:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885785:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.885789:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885803:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885820:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885835:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a509c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885856:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.885859:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885865:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.885868:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885876:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885887:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885894:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5064 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7255:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885911:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.885915:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885921:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.885924:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885931:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885941:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885950:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee522aef/0xf62499ab927a508e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7257:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885963:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.885966:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885970:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.885972:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885980:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885986:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885990:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5080 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.885998:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.886000:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886004:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.886006:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886010:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886014:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886018:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a5087 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886030:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.886032:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886036:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.886038:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886042:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886046:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886050:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e0e7058/0xf62499ab927a5079 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886057:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.886058:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.886059:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.886065:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.886073:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886077:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.886277:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1252:3655:x1871938510691968:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15714us (16060us total) trans 4295070624 rc 0/0 00000100:00100000:0.0:1785220581.886285:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284438 00010000:00010000:0.0:1785220581.886307:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.886328:0:14191:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a43c83b/0xf62499ab9278f2fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ae expref: 225 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.886336:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ec380/2, rc 0, transno 4295070623 00000100:00100000:3.0:1785220581.886339:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0000_UUID:14191:1871938510693888:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.886348:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510693504:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220581.886351:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886359:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886364:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886372:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886378:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886391:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1785220581.886417:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98405880/2, rc 0, transno 4295070038 00000100:00100000:0.0:1785220581.886426:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98405880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510692864:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.886436:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac8e00/2, rc 0, transno 4295070625 00000100:00100000:0.0:1785220581.886442:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510691840:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.886474:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1252:3652:x1871938510693632:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13952us (14136us total) trans 4295070627 rc 0/0 00000100:00100000:0.0:1785220581.886477:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510694656:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.886480:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284443 00000100:00100000:0.0:1785220581.886532:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.886566:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ed880/2, rc 0, transno 4295070627 00000100:00100000:3.0:1785220581.886570:0:14168:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510694656 00000100:00100000:0.0:1785220581.886574:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510693632:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.886575:0:14168:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510694656 00000100:00100000:3.0:1785220581.886607:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284447 00000100:00100000:3.0:1785220581.886615:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1252:3652:x1871938510694656:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.886632:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.886638:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29325 count 1 00010000:00010000:1.0:1785220581.886641:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.886653:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.886656:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886669:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e57593f8/0xf62499ab9278f36e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f11e expref: 224 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886677:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x728d:0x0].0x0 since=21s v=(0 0) 00010000:00010000:3.0:1785220581.886682:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.886686:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 4/0,1 mode: --/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.886711:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e57593f8/0xf62499ab9278f36e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f11e expref: 224 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.886718:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1253:3655:x1871938510690048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23982us (28749us total) trans 4295070628 rc 0/0 00010000:00010000:3.0:1785220581.886729:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e57593f8/0xf62499ab9278f36e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f11e expref: 224 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.886740:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510694912:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.886790:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f480 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.886795:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98406d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510693248:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.886810:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.886899:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510694784:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.886918:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.886949:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510694784 00000100:00100000:3.0:1785220581.886954:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510694784 00000100:00100000:0.0:1785220581.886977:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.886978:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284448 00000100:00100000:3.0:1785220581.886984:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+1254:3653:x1871938510694784:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.886995:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29326 count 1 00010000:00010000:3.0:1785220581.887006:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887017:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001a9fa744/0xf62499ab9278f3a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f156 expref: 224 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887024:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x728e:0x0].0x0 since=21s v=(0 0) 00010000:00010000:3.0:1785220581.887027:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.887030:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.887045:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.887046:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001a9fa744/0xf62499ab9278f3a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f156 expref: 224 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887061:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001a9fa744/0xf62499ab9278f3a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f156 expref: 224 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.887071:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510695040:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.887076:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.887131:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955584 00000100:00100000:2.0:1785220581.887136:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955584 00010000:00010000:2.0:1785220581.887156:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000923f53d2/0xf62499ab9278c3ef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dab9 expref: 269 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.887178:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955456 00000100:00100000:2.0:1785220581.887181:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955456 00010000:00010000:2.0:1785220581.887194:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005aaaa02c/0xf62499ab9278c500 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db22 expref: 270 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.887210:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519953792 00000100:00100000:2.0:1785220581.887212:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519953792 00010000:00010000:2.0:1785220581.887225:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cae6b937/0xf62499ab9278f088 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef42 expref: 225 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.887245:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100373 00000100:00100000:2.0:1785220581.887252:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89effb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+270:2404:x1871938519955584:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.887261:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.887266:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000923f53d2/0xf62499ab9278c3ef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dab9 expref: 270 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.887277:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.887277:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000923f53d2/0xf62499ab9278c3ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dab9 expref: 270 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.887294:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000923f53d2/0xf62499ab9278c3ef lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dab9 expref: 270 pid: 14168 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.887296:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100374 00000100:00100000:3.0:1785220581.887304:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+269:2407:x1871938519955456:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.887305:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7258:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.887308:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7258:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.887311:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.887314:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887315:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005aaaa02c/0xf62499ab9278c500 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db22 expref: 269 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.887323:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887325:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005aaaa02c/0xf62499ab9278c500 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db22 expref: 269 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887338:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005aaaa02c/0xf62499ab9278c500 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7db22 expref: 269 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.887341:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220581.887347:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7259:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.887350:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7259:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.887354:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887361:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.887378:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.887384:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284430 00010000:00010000:1.0:1785220581.887413:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.887421:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.887429:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.887436:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.887449:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.887542:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100375 00000100:00100000:0.0:1785220581.887549:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+225:2407:x1871938519953792:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.887557:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.887561:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cae6b937/0xf62499ab9278f088 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef42 expref: 225 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887570:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cae6b937/0xf62499ab9278f088 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef42 expref: 225 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887582:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cae6b937/0xf62499ab9278f088 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef42 expref: 225 pid: 14190 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887591:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7285:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.887594:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7285:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.887598:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887605:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887620:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.887632:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+224:2407:x1871938519953792:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (555us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.887637:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100375 00010000:00010000:0.0:1785220581.887655:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887663:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887669:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887676:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.887683:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.887744:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1188:3655:x1871938510692992:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14369us (14987us total) trans 4295070040 rc 0/0 00000100:00100000:0.0:1785220581.887750:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284445 00000100:00100000:0.0:1785220581.887810:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.887851:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.887886:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.887914:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954048 00000100:00100000:0.0:1785220581.887918:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954048 00010000:00010000:0.0:1785220581.887934:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000051f57152/0xf62499ab9278bee7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d858 expref: 269 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.887967:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+269:2407:x1871938519955456:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 664us (920us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.887967:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89effb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+269:2404:x1871938519955584:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 716us (990us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.887974:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100374 00000100:00100000:3.0:1785220581.887987:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954304 00000100:00100000:3.0:1785220581.887990:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954304 00000100:00100000:0.0:1785220581.887990:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519954560 00000100:00100000:0.0:1785220581.887992:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519954560 00010000:00010000:3.0:1785220581.888004:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000050d15215/0xf62499ab9278bfa4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8cf expref: 270 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888004:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004b3d7b96/0xf62499ab9278c673 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dba0 expref: 270 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.888077:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100373 00000100:00100000:2.0:1785220581.888109:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100376 00000100:00100000:2.0:1785220581.888116:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+269:2407:x1871938519954048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.888124:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.888129:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000051f57152/0xf62499ab9278bee7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d858 expref: 269 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888138:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000051f57152/0xf62499ab9278bee7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d858 expref: 269 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888152:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000051f57152/0xf62499ab9278bee7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d858 expref: 269 pid: 14260 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888161:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7253:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.888164:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7253:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.888168:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888175:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888199:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.888215:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+268:2407:x1871938519954048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (405us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.888222:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100376 00010000:00010000:2.0:1785220581.888242:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888250:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888259:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888266:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.888274:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:0.0:1785220581.888338:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100377 00000100:00100000:3.0:1785220581.888342:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100378 00000100:00100000:0.0:1785220581.888345:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+267:2407:x1871938519954560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.888349:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+267:2407:x1871938519954304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.888353:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.888355:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.888358:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004b3d7b96/0xf62499ab9278c673 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dba0 expref: 267 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888359:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000050d15215/0xf62499ab9278bfa4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8cf expref: 267 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888370:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000050d15215/0xf62499ab9278bfa4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8cf expref: 267 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888370:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004b3d7b96/0xf62499ab9278c673 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dba0 expref: 267 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888385:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000050d15215/0xf62499ab9278bfa4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8cf expref: 267 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888385:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004b3d7b96/0xf62499ab9278c673 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dba0 expref: 267 pid: 14227 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888395:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.888396:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7254:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.888398:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.888399:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7254:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.888403:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888403:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888413:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888413:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888429:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220581.888430:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.888443:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+265:2407:x1871938519954560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (557us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.888444:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+265:2407:x1871938519954304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (593us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.888449:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100377 00000100:00100000:3.0:1785220581.888452:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100378 00010000:00010000:0.0:1785220581.888467:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888470:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888476:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888479:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888484:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888486:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.888492:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888494:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.888501:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/7 locks to regular blp list, 7 blwis in pool 00010000:00010000:0.0:1785220581.888501:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:3.0:1785220581.888583:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1189:3655:x1871938510691072:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22677us (23244us total) trans 4295070041 rc 0/0 00000100:00100000:1.0:1785220581.888584:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1189:3655:x1871938510693376:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15536us (15869us total) trans 4295070039 rc 0/0 00000100:00100000:3.0:1785220581.888590:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284435 00000100:00100000:1.0:1785220581.888593:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284444 00010000:00010000:1.0:1785220581.888610:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888619:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888626:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888634:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888643:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/8 locks to regular blp list, 8 blwis in pool 00000100:00100000:1.0:1785220581.888710:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+1188:3653:x1871938510690944:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23228us (23542us total) trans 4295070043 rc 0/0 00000100:00100000:1.0:1785220581.888716:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284433 00010000:00010000:1.0:1785220581.888736:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 7 total in blp. (7 blwis in pool) 00010000:00010000:1.0:1785220581.888741:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888746:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.888749:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888767:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888785:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888793:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a50a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888810:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 6 total in blp. (6 blwis in pool) 00010000:00010000:1.0:1785220581.888814:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.888821:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.888824:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.888928:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1254:3653:x1871938510690688:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24101us (24187us total) trans 4295070629 rc 0/0 00000100:00100000:0.0:1785220581.888928:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1187:3654:x1871938510694272:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13780us (13907us total) trans 4295070042 rc 0/0 00000100:00100000:0.0:1785220581.888934:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284446 00000100:00100000:2.0:1785220581.888944:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284432 00000100:00100000:2.0:1785220581.888966:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510694528:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.888981:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac8700/2, rc 0, transno 4295070043 00000100:00100000:0.0:1785220581.888987:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510690944:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.888997:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.888997:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac8380/2, rc 0, transno 4295070629 00000100:00100000:0.0:1785220581.889006:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510690688:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.889031:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98406300/2, rc 0, transno 4295070039 00000100:00100000:2.0:1785220581.889035:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98406300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510693376:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.889045:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98407100/2, rc 0, transno 4295070040 00000100:00100000:2.0:1785220581.889048:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98407100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510692992:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.889057:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac9c00/2, rc 0, transno 4295070624 00000100:00100000:2.0:1785220581.889071:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510691968:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.889081:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083f800/2, rc 0, transno 4295070626 00000100:00100000:0.0:1785220581.889145:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.889154:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510695424:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.889178:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.889202:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510694528 00000100:00100000:0.0:1785220581.889206:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510694528 00000100:00100000:0.0:1785220581.889227:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510695424 00000100:00100000:0.0:1785220581.889230:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510695424 00000100:00100000:0.0:1785220581.889245:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284449 00000100:00100000:0.0:1785220581.889251:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90e5c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1255:3655:x1871938510694528:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.889268:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29324 count 1 00010000:00010000:0.0:1785220581.889285:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889299:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000074c8dfc4/0xf62499ab9278f336 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ed expref: 223 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889306:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x728c:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220581.889309:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.889312:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889331:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000074c8dfc4/0xf62499ab9278f336 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ed expref: 223 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889343:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000074c8dfc4/0xf62499ab9278f336 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ed expref: 223 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.889354:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510695552:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.889384:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.889401:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284450 00000100:00100000:0.0:1785220581.889408:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1255:3653:x1871938510695424:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.889418:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29329 count 1 00010000:00010000:0.0:1785220581.889429:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889441:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000093bcfe34/0xf62499ab9278f44e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1c6 expref: 223 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889447:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7291:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220581.889450:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7291:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.889454:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889471:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000093bcfe34/0xf62499ab9278f44e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1c6 expref: 223 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.889480:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000093bcfe34/0xf62499ab9278f44e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1c6 expref: 223 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.889490:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510695680:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.889511:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.889580:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.889590:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510692096:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.889607:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac9500/2, rc 0, transno 4295070041 00000100:00100000:2.0:1785220581.889611:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510691072:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.889621:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac9880/2, rc 0, transno 4295070037 00000100:00100000:2.0:1785220581.889624:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510691200:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.889634:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94acb480/2, rc 0, transno 4295070628 00000100:00100000:3.0:1785220581.889670:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955200 00000100:00100000:3.0:1785220581.889674:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955200 00010000:00010000:3.0:1785220581.889696:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000161b1c26/0xf62499ab9278c20c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d9d2 expref: 264 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889697:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889710:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889718:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000096fcc85/0xf62499ab927a5048 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.889726:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510690048:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.889735:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00000100:00100000:2.0:1785220581.889737:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510695296:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.889739:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889746:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.889749:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889768:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.889770:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.889779:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889786:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c64f6865/0xf62499ab927a50aa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7259:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.889788:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.889793:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510695168:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.889800:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220581.889804:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.889811:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220581.889811:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.889814:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889821:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889901:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889909:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a50b1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7258:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889927:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.889931:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889937:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.889940:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889948:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889956:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889965:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a5056 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889976:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.889979:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889985:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.889988:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.889995:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890002:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890008:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2eba103/0xf62499ab927a50b8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890018:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.890020:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890026:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.890028:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890034:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890042:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890049:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a506b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7254:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890058:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.890061:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890067:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.890069:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890076:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890083:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890107:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a505d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7253:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890118:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.890120:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.890121:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.890131:0:14206:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510695296 00000100:00100000:1.0:1785220581.890135:0:14206:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510695296 00000100:00100000:1.0:1785220581.890161:0:14206:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510695168 00000100:00100000:1.0:1785220581.890164:0:14206:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510695168 00000100:00100000:1.0:1785220581.890182:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284451 00000100:00100000:1.0:1785220581.890188:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1257:3655:x1871938510695296:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.890201:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29328 count 1 00010000:00010000:1.0:1785220581.890214:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.890216:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284452 00000100:00100000:2.0:1785220581.890223:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1257:3655:x1871938510695168:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.890227:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000041ab3a22/0xf62499ab9278f440 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b8 expref: 223 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.890234:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29327 count 1 00010000:00010000:1.0:1785220581.890234:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7290:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.890237:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7290:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.890241:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890244:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890254:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003f816a19/0xf62499ab9278f41d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1aa expref: 223 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890260:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000041ab3a22/0xf62499ab9278f440 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b8 expref: 223 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890262:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x728f:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220581.890265:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.890268:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890269:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000041ab3a22/0xf62499ab9278f440 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b8 expref: 223 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.890281:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98405f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510695808:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.890284:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003f816a19/0xf62499ab9278f41d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1aa expref: 223 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890293:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003f816a19/0xf62499ab9278f41d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1aa expref: 223 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.890301:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510695936:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.890306:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.890316:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01840 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.890323:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.890336:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890349:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890363:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890373:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890391:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.890430:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.890435:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890441:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.890444:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890451:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890466:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890473:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a5072 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.890486:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.890487:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.890489:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.890539:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100379 00000100:00100000:3.0:1785220581.890547:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+264:2407:x1871938519955200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.890556:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.890562:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000161b1c26/0xf62499ab9278c20c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d9d2 expref: 264 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.890572:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000161b1c26/0xf62499ab9278c20c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d9d2 expref: 264 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.890575:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1258:3653:x1871938510691712:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21470us (21605us total) trans 4295070630 rc 0/0 00000100:00100000:1.0:1785220581.890581:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284436 00010000:00010000:3.0:1785220581.890587:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000161b1c26/0xf62499ab9278c20c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d9d2 expref: 264 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.890598:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7256:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.890600:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.890601:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7256:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.890606:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.890607:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.890614:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.890673:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.890689:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+263:2407:x1871938519955200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 143us (1109us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.890699:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100379 00000100:00100000:0.0:1785220581.890721:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510696064:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.890745:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.890780:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510696064 00000100:00100000:3.0:1785220581.890784:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510696064 00000004:00080000:0.0:1785220581.890788:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac8000/2, rc 0, transno 4295070630 00000100:00100000:0.0:1785220581.890797:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510691712:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.890804:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220581.890806:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284453 00000100:00100000:0.0:1785220581.890810:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98404e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510696448:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.890812:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1258:3653:x1871938510696064:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220581.890891:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29330 count 1 00010000:00010000:3.0:1785220581.890905:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.890918:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e57f6b65/0xf62499ab9278f45c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1d4 expref: 223 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.890923:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:3.0:1785220581.890925:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7292:0x0].0x0 since=21s v=(0 0) 00010000:00010000:3.0:1785220581.890928:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7292:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.890931:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98407b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510696576:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.890932:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.890951:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e57f6b65/0xf62499ab9278f45c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1d4 expref: 223 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.890954:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.890961:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e57f6b65/0xf62499ab9278f45c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1d4 expref: 223 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.890971:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510696960:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.890973:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510696576 00000100:00100000:1.0:1785220581.890976:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510696576 00000100:00100000:3.0:1785220581.890990:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.890998:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284454 00000100:00100000:1.0:1785220581.891004:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98406680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1187:3652:x1871938510696576:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.891015:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29278 count 1 00010000:00010000:2.0:1785220581.891019:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891028:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.891029:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.891038:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891040:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000500f4f6e/0xf62499ab9278c9de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd44 expref: 262 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.891048:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891053:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x725e:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.891056:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.891060:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.891061:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.891076:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000500f4f6e/0xf62499ab9278c9de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd44 expref: 262 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891103:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000500f4f6e/0xf62499ab9278c9de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd44 expref: 262 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.891115:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98407480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510697088:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.891134:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777b40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.891148:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.891153:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891160:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.891163:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891171:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891188:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891195:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5095 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7256:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891212:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.891214:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.891216:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.891241:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98405c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696320:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.891266:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.891286:0:14164:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510696320 00000100:00100000:1.0:1785220581.891289:0:14164:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510696320 00000100:00100000:2.0:1785220581.891295:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1189:3653:x1871938510692736:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19271us (19631us total) trans 4295070044 rc 0/0 00000100:00100000:2.0:1785220581.891301:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284441 00000100:00100000:2.0:1785220581.891314:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284455 00000100:00100000:2.0:1785220581.891321:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98406d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1188:3655:x1871938510696320:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.891326:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98404a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696832:0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.891331:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29276 count 1 00010000:00010000:2.0:1785220581.891342:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.891347:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.891354:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004391f86a/0xf62499ab9278c8a3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc80 expref: 262 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.891361:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x725c:0x0].0x0 since=24s v=(0 0) 00000100:00100000:1.0:1785220581.891362:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00010000:00010000:2.0:1785220581.891364:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725c:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.891367:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98407800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696192:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.891368:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.891385:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004391f86a/0xf62499ab9278c8a3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc80 expref: 262 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.891385:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.891394:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004391f86a/0xf62499ab9278c8a3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc80 expref: 262 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.891403:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98404700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696704:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.891404:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510697216:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.891446:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510696832 00000100:00100000:3.0:1785220581.891450:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510696832 00000100:00100000:1.0:1785220581.891470:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.891477:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510696192 00000100:00100000:3.0:1785220581.891480:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510696192 00000100:00100000:3.0:1785220581.891499:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510696704 00000100:00100000:3.0:1785220581.891502:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510696704 00000100:00100000:1.0:1785220581.891508:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284456 00000100:00100000:1.0:1785220581.891515:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98406a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1191:3655:x1871938510696832:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.891520:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284457 00000100:00100000:3.0:1785220581.891525:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98405180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1191:3655:x1871938510696192:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.891527:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29280 count 1 00002000:00080000:3.0:1785220581.891534:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29275 count 1 00010000:00010000:1.0:1785220581.891537:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.891544:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891549:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007fb711fd/0xf62499ab9278cbcf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de08 expref: 262 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.891554:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005f778c18/0xf62499ab9278c768 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc02 expref: 262 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891555:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7260:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.891559:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7260:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.891562:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x725b:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.891562:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.891564:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.891568:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.891581:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005f778c18/0xf62499ab9278c768 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc02 expref: 262 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891581:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007fb711fd/0xf62499ab9278cbcf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de08 expref: 262 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.891592:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005f778c18/0xf62499ab9278c768 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc02 expref: 262 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891592:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007fb711fd/0xf62499ab9278cbcf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de08 expref: 262 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.891602:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510697472:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.891602:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510697344:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.891632:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284458 00000100:00100000:1.0:1785220581.891640:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98404000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1191:3655:x1871938510696704:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.891641:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00002000:00080000:1.0:1785220581.891672:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29279 count 1 00000100:00100000:3.0:1785220581.891679:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.891683:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891694:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005259ddac/0xf62499ab9278ca9b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddb4 expref: 262 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891707:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x725f:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.891710:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.891714:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891732:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005259ddac/0xf62499ab9278ca9b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddb4 expref: 262 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.891741:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005259ddac/0xf62499ab9278ca9b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddb4 expref: 262 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.891751:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510697600:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.891779:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.891817:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.891888:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.891957:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510696448 00000100:00100000:1.0:1785220581.891961:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510696448 00000100:00100000:1.0:1785220581.891984:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284459 00000100:00100000:1.0:1785220581.891990:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1192:3653:x1871938510696448:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.892002:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29277 count 1 00010000:00010000:1.0:1785220581.892013:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892025:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bdd42969/0xf62499ab9278c928 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dce2 expref: 262 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892037:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x725d:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.892040:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.892044:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892059:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bdd42969/0xf62499ab9278c928 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dce2 expref: 262 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892068:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bdd42969/0xf62499ab9278c928 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dce2 expref: 262 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.892077:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510697728:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.892121:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9aedcd80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.892217:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510697856:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.892243:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220581.892299:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98405500/2, rc 0, transno 4295070044 00000100:00100000:0.0:1785220581.892310:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98405500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510692736:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.892355:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.892361:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510697984:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.892383:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.892402:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510697856 00000100:00100000:1.0:1785220581.892406:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510697856 00000100:00100000:1.0:1785220581.892431:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510697984 00000100:00100000:1.0:1785220581.892434:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510697984 00000100:00100000:1.0:1785220581.892449:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284460 00000100:00100000:1.0:1785220581.892455:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1259:3652:x1871938510697856:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.892468:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29331 count 1 00010000:00010000:1.0:1785220581.892481:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892495:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e3212baf/0xf62499ab9278f504 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1e9 expref: 223 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892501:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7293:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.892504:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7293:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.892549:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284461 00000100:00100000:3.0:1785220581.892559:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1193:3652:x1871938510697984:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.892573:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29281 count 1 00010000:00010000:3.0:1785220581.892585:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.892596:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000051797346/0xf62499ab9278cc93 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de7f expref: 262 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.892602:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7261:0x0].0x0 since=24s v=(0 0) 00010000:00010000:3.0:1785220581.892605:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7261:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.892609:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.892628:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000051797346/0xf62499ab9278cc93 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de7f expref: 262 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892662:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892685:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e3212baf/0xf62499ab9278f504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1e9 expref: 223 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.892695:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e3212baf/0xf62499ab9278f504 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1e9 expref: 223 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.892707:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510698240:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.892725:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000051797346/0xf62499ab9278cc93 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de7f expref: 262 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.892735:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510698112:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.892735:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdb40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.892773:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f840 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.892800:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98404380/2, rc 0, transno 4295070042 00000100:00100000:3.0:1785220581.892804:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98404380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510694272:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.893022:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510698368:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.893049:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.893071:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510698368 00000100:00100000:1.0:1785220581.893075:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510698368 00000100:00100000:1.0:1785220581.893220:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284462 00000100:00100000:1.0:1785220581.893227:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1194:3654:x1871938510698368:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.893240:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29282 count 1 00010000:00010000:1.0:1785220581.893252:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.893263:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004966ee5a/0xf62499ab9278ce06 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dee8 expref: 262 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.893269:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7262:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.893271:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7262:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.893275:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.893291:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004966ee5a/0xf62499ab9278ce06 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dee8 expref: 262 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.893298:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004966ee5a/0xf62499ab9278ce06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dee8 expref: 262 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.893307:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510698496:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.893332:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.895530:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510695040:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.895560:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.895915:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510694912:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.895933:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.896603:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510695680:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.896624:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.896662:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.896792:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510695552:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.896808:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.896867:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.901488:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955840 00000100:00100000:1.0:1785220581.901493:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956096 00000100:00100000:3.0:1785220581.901502:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955840 00000100:00100000:1.0:1785220581.901502:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956096 00010000:00010000:1.0:1785220581.901547:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000093bcfe34/0xf62499ab9278f44e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1c6 expref: 225 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.901591:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100380 00000100:00100000:1.0:1785220581.901600:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+225:2407:x1871938519956096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.901617:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.901623:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000093bcfe34/0xf62499ab9278f44e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1c6 expref: 225 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901638:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000093bcfe34/0xf62499ab9278f44e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1c6 expref: 225 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901661:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093bcfe34/0xf62499ab9278f44e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1c6 expref: 225 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901677:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7291:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.901680:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7291:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.901685:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901691:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901721:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.901746:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.901772:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.901782:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.901790:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.901795:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+224:2407:x1871938519956096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 196us (4929us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.901803:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100380 00010000:00010000:2.0:1785220581.901808:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.901834:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001a9fa744/0xf62499ab9278f3a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f156 expref: 223 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.901858:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100381 00000100:00100000:1.0:1785220581.901865:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+223:2407:x1871938519955840:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.901873:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.901878:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001a9fa744/0xf62499ab9278f3a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f156 expref: 223 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901888:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001a9fa744/0xf62499ab9278f3a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f156 expref: 223 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901901:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a9fa744/0xf62499ab9278f3a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f156 expref: 223 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901911:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.901913:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.901916:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901928:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.901944:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.901962:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2407:x1871938519955840:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (5302us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.901969:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100381 00010000:00010000:1.0:1785220581.901990:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.902015:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.902023:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.902030:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.902042:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220581.903685:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220581.903703:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903723:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.903730:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903746:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903781:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903794:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a50d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903840:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.903847:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903867:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.903871:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903885:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903901:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903913:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a50c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.903936:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.903938:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.903942:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.904228:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0ca80/2, rc 0, transno 4295070631 00000100:00100000:0.0:1785220581.904245:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510695424:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.904314:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000004:00080000:0.0:1785220581.904317:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efc380/2, rc 0, transno 4295070632 00000100:00100000:0.0:1785220581.904323:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510694784:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.904329:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510698624:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.904349:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.904356:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510698752:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220581.904377:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.904404:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.904419:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510698624 00000100:00100000:3.0:1785220581.904425:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510698624 00000100:00100000:0.0:1785220581.904446:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.904460:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510698752 00000100:00100000:3.0:1785220581.904463:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510698752 00000100:00100000:0.0:1785220581.904465:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.904486:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284463 00000100:00100000:3.0:1785220581.904494:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1263:3654:x1871938510698624:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.904507:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00002000:00080000:3.0:1785220581.904517:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29332 count 1 00000100:00100000:0.0:1785220581.904521:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.904537:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.904554:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000db928590/0xf62499ab9278f519 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f7 expref: 221 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.904562:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7294:0x0].0x0 since=21s v=(0 0) 00010000:00010000:3.0:1785220581.904567:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7294:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.904572:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.904575:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.904591:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.904596:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000db928590/0xf62499ab9278f519 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f7 expref: 221 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.904607:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955712 00010000:00010000:3.0:1785220581.904608:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000db928590/0xf62499ab9278f519 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f7 expref: 221 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.904610:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955712 00000100:00100000:3.0:1785220581.904620:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510698880:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.904629:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e57593f8/0xf62499ab9278f36e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f11e expref: 222 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.904647:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519955968 00000100:00100000:0.0:1785220581.904650:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519955968 00010000:00010000:0.0:1785220581.904661:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000074c8dfc4/0xf62499ab9278f336 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ed expref: 223 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.904668:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.904676:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956352 00000100:00100000:0.0:1785220581.904678:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956352 00000100:00100000:3.0:1785220581.904689:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510696960:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.904689:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003f816a19/0xf62499ab9278f41d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1aa expref: 224 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.904704:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956224 00000100:00100000:0.0:1785220581.904707:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956224 00010000:00010000:3.0:1785220581.904708:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904718:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000041ab3a22/0xf62499ab9278f440 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b8 expref: 225 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.904727:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284464 00000100:00100000:0.0:1785220581.904732:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956480 00000100:00100000:3.0:1785220581.904734:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510698752:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.904734:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956480 00010000:00010000:0.0:1785220581.904743:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e57f6b65/0xf62499ab9278f45c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1d4 expref: 226 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.904748:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29333 count 1 00010000:00010000:3.0:1785220581.904772:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.904773:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956736 00000100:00100000:0.0:1785220581.904776:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956736 00010000:00010000:3.0:1785220581.904784:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000630b2488/0xf62499ab9278f527 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f205 expref: 226 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904789:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004391f86a/0xf62499ab9278c8a3 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc80 expref: 263 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.904790:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7295:0x0].0x0 since=20s v=(0 0) 00010000:00010000:3.0:1785220581.904793:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7295:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.904797:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.904808:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100382 00000100:00100000:0.0:1785220581.904813:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+226:2406:x1871938519955712:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.904814:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000630b2488/0xf62499ab9278f527 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f205 expref: 226 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904822:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.904826:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e57593f8/0xf62499ab9278f36e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f11e expref: 226 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904837:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e57593f8/0xf62499ab9278f36e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f11e expref: 226 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904853:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e57593f8/0xf62499ab9278f36e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f11e expref: 226 pid: 14204 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904864:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.904867:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.904872:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904882:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904906:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.904928:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+225:2406:x1871938519955712:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (483us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.904936:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100382 00010000:00010000:3.0:1785220581.904942:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000630b2488/0xf62499ab9278f527 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f205 expref: 225 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.904953:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510699008:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.904954:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100383 00000100:00100000:0.0:1785220581.904960:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+224:2406:x1871938519955968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.904970:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220581.904975:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f240 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.904975:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000074c8dfc4/0xf62499ab9278f336 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ed expref: 224 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904984:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000074c8dfc4/0xf62499ab9278f336 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ed expref: 224 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.904989:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.904996:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c8dfc4/0xf62499ab9278f336 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0ed expref: 224 pid: 14258 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.904998:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.905007:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.905010:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.905012:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.905014:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905021:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.905022:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905036:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220581.905042:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.905059:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+223:2406:x1871938519955968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (593us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.905065:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100383 00000100:00100000:0.0:1785220581.905078:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100384 00000100:00100000:2.0:1785220581.905214:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510695424:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15804us (16033us total) trans 4295070631 rc 0/0 00000100:00100000:2.0:1785220581.905222:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284450 00000100:00100000:0.0:1785220581.905254:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2407:x1871938519956352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.905259:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510695936:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.905264:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.905269:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003f816a19/0xf62499ab9278f41d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1aa expref: 222 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.905271:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.905280:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003f816a19/0xf62499ab9278f41d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1aa expref: 222 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.905343:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510694784:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18357us (18421us total) trans 4295070632 rc 0/0 00000100:00100000:1.0:1785220581.905350:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284448 00000100:00100000:3.0:1785220581.905361:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1263:3652:x1871938510694656:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18747us (18829us total) trans 4295070633 rc 0/0 00000100:00100000:3.0:1785220581.905366:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284447 00000100:00100000:3.0:1785220581.905386:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100385 00000100:00100000:3.0:1785220581.905392:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2407:x1871938519956224:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.905399:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.905404:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000041ab3a22/0xf62499ab9278f440 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b8 expref: 222 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905412:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000041ab3a22/0xf62499ab9278f440 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b8 expref: 222 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905423:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7290:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.905425:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7290:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.905429:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905440:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905450:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.905468:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2407:x1871938519956224:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (945us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.905474:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100385 00000100:00100000:3.0:1785220581.905487:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100386 00000100:00100000:3.0:1785220581.905493:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+221:2406:x1871938519956480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.905500:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.905504:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e57f6b65/0xf62499ab9278f45c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1d4 expref: 221 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905513:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e57f6b65/0xf62499ab9278f45c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1d4 expref: 221 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905525:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e57f6b65/0xf62499ab9278f45c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1d4 expref: 221 pid: 14179 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905535:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7292:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.905538:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7292:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.905542:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905553:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905568:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.905582:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+220:2406:x1871938519956480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (1008us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.905588:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100386 00000100:00100000:3.0:1785220581.905600:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100387 00000100:00100000:3.0:1785220581.905606:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+263:2406:x1871938519956736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.905613:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.905617:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004391f86a/0xf62499ab9278c8a3 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc80 expref: 263 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905627:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004391f86a/0xf62499ab9278c8a3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc80 expref: 263 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905640:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.905643:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.905647:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905655:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905670:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.905684:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+263:2406:x1871938519956736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (1092us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.905689:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100387 00010000:00010000:3.0:1785220581.905704:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905712:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905720:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905726:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.905738:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.905962:0:8440:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004391f86a/0xf62499ab9278c8a3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc80 expref: 262 pid: 14144 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.905973:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510697216:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.905987:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.906000:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.906007:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.906015:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.906021:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.906037:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220581.906226:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98406d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1195:3655:x1871938510696320:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14905us (14959us total) trans 4295070045 rc 0/0 00010000:00010000:0.0:1785220581.906227:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f816a19/0xf62499ab9278f41d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1aa expref: 219 pid: 14195 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.906233:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284455 00010000:00010000:0.0:1785220581.906240:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906242:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x728f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906246:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906254:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.906276:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.906277:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.906296:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+218:2407:x1871938519956352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1212us (1788us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.906301:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100384 00000100:00100000:0.0:1785220581.906330:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956608 00000100:00100000:0.0:1785220581.906335:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956608 00000100:00100000:2.0:1785220581.906348:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510697344:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.906350:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000500f4f6e/0xf62499ab9278c9de lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd44 expref: 262 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.906366:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.906370:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100388 00000100:00100000:0.0:1785220581.906377:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+262:2406:x1871938519956608:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.906384:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.906388:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000500f4f6e/0xf62499ab9278c9de lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd44 expref: 262 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906397:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000500f4f6e/0xf62499ab9278c9de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd44 expref: 262 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906410:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906413:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906417:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906425:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906437:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.906444:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.906453:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+262:2406:x1871938519956608:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.906460:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100388 00000100:00100000:0.0:1785220581.906470:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956864 00000100:00100000:2.0:1785220581.906471:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.906472:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956864 00010000:00010000:0.0:1785220581.906484:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007fb711fd/0xf62499ab9278cbcf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de08 expref: 262 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.906501:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519956992 00000100:00100000:0.0:1785220581.906504:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519956992 00000100:00100000:2.0:1785220581.906515:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.906515:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005f778c18/0xf62499ab9278c768 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc02 expref: 263 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.906525:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100389 00000100:00100000:0.0:1785220581.906531:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+263:2407:x1871938519956864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.906537:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.906542:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007fb711fd/0xf62499ab9278cbcf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de08 expref: 263 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906551:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007fb711fd/0xf62499ab9278cbcf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de08 expref: 263 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906562:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007fb711fd/0xf62499ab9278cbcf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de08 expref: 263 pid: 14183 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906576:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7260:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906578:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7260:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906582:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906589:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906610:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.906629:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+262:2407:x1871938519956864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (184us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.906636:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100389 00000100:00100000:0.0:1785220581.906648:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519957120 00000100:00100000:0.0:1785220581.906651:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519957120 00010000:00010000:0.0:1785220581.906663:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005259ddac/0xf62499ab9278ca9b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddb4 expref: 262 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.906676:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100390 00000100:00100000:0.0:1785220581.906681:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+262:2407:x1871938519956992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.906687:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.906692:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005f778c18/0xf62499ab9278c768 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc02 expref: 262 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906700:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005f778c18/0xf62499ab9278c768 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc02 expref: 262 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906713:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906716:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906720:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906727:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906740:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.906769:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+262:2407:x1871938519956992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 87us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.906776:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100390 00000100:00100000:0.0:1785220581.906788:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100391 00000100:00100000:0.0:1785220581.906795:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+261:2407:x1871938519957120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.906802:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.906807:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005259ddac/0xf62499ab9278ca9b lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddb4 expref: 261 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906816:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005259ddac/0xf62499ab9278ca9b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddb4 expref: 261 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906828:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906831:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.906835:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906843:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906853:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.906870:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+261:2407:x1871938519957120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.906877:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100391 00010000:00010000:0.0:1785220581.906895:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906905:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906914:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906921:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.906931:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220581.907011:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510698112:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.907026:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.908443:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efe300/2, rc 0, transno 4295070633 00000100:00100000:0.0:1785220581.908465:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510694656:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.908535:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510699136:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.908583:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.908617:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510699136 00000100:00100000:0.0:1785220581.908623:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510699136 00000100:00100000:0.0:1785220581.908663:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284465 00000100:00100000:0.0:1785220581.908671:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1262:3653:x1871938510699136:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.908693:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29334 count 1 00010000:00010000:0.0:1785220581.908715:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.908735:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee1023ff/0xf62499ab9278f53c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f213 expref: 217 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.908745:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7296:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220581.908748:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7296:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.908753:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.908788:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ee1023ff/0xf62499ab9278f53c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f213 expref: 217 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.908800:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ee1023ff/0xf62499ab9278f53c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f213 expref: 217 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.908812:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510699264:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.908861:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9900 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.908912:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98405c00/2, rc 0, transno 4295070045 00000100:00100000:0.0:1785220581.908923:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98405c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696320:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.908968:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:2.0:1785220581.909282:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.909288:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510699392:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.909293:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.909303:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.909311:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.909316:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.909322:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:0.0:1785220581.909433:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.909451:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519957376 00000100:00100000:0.0:1785220581.909455:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519957376 00000100:00100000:2.0:1785220581.909468:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98406a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1195:3655:x1871938510696832:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17953us (18120us total) trans 4295070046 rc 0/0 00010000:00010000:0.0:1785220581.909474:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000051797346/0xf62499ab9278cc93 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de7f expref: 261 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.909475:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284456 00000100:00100000:2.0:1785220581.909492:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510699392 00000100:00100000:0.0:1785220581.909494:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100392 00000100:00100000:2.0:1785220581.909495:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510699392 00000100:00100000:0.0:1785220581.909500:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+261:2407:x1871938519957376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.909509:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220581.909513:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284466 00010000:00010000:0.0:1785220581.909515:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000051797346/0xf62499ab9278cc93 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de7f expref: 261 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.909518:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1195:3655:x1871938510699392:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.909526:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000051797346/0xf62499ab9278cc93 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de7f expref: 261 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.909529:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29283 count 1 00010000:00010000:2.0:1785220581.909542:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.909544:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000051797346/0xf62499ab9278cc93 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de7f expref: 261 pid: 14252 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.909553:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cd2b1663/0xf62499ab9278cf4f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df7b expref: 260 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.909558:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7261:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.909561:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7261:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.909566:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7263:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.909566:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.909569:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7263:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.909573:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.909581:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.909592:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cd2b1663/0xf62499ab9278cf4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df7b expref: 260 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.909598:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.909601:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cd2b1663/0xf62499ab9278cf4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df7b expref: 260 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.909611:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510699520:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.909615:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+260:2407:x1871938519957376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.909621:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100392 00000100:00100000:2.0:1785220581.909636:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01cc0 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.909656:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98404a80/2, rc 0, transno 4295070046 00000100:00100000:2.0:1785220581.909660:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.909664:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98404a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696832:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.909680:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519957632 00000100:00100000:0.0:1785220581.909684:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519957632 00000100:00100000:2.0:1785220581.909689:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.909698:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bdd42969/0xf62499ab9278c928 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dce2 expref: 260 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.909706:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.909715:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519957248 00000100:00100000:0.0:1785220581.909718:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519957248 00010000:00010000:0.0:1785220581.909730:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e3212baf/0xf62499ab9278f504 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1e9 expref: 218 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.909744:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519957504 00000100:00100000:0.0:1785220581.909747:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519957504 00010000:00010000:2.0:1785220581.910203:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1785220581.910211:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910219:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.910224:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910242:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910256:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 1/0,0 mode: --/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910265:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a50bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910285:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220581.910290:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910297:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.910300:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910308:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910316:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910322:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a50e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910339:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.910342:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910348:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.910350:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910356:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910363:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910370:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a50f7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910382:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.910386:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910393:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.910397:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910411:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910420:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910429:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a50e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910443:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.910449:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910455:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.910459:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910467:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910476:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910484:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d7fb77d/0xf62499ab927a50fe lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7260:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910501:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.910503:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.910505:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.910524:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100393 00000100:00100000:2.0:1785220581.910532:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+261:2407:x1871938519957632:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.910542:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.910548:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bdd42969/0xf62499ab9278c928 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dce2 expref: 261 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910560:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bdd42969/0xf62499ab9278c928 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dce2 expref: 261 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910574:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.910577:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x725d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.910582:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910592:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910604:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.910630:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+261:2407:x1871938519957632:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (968us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.910637:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100393 00000100:00100000:2.0:1785220581.910676:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100394 00000100:00100000:2.0:1785220581.910683:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+218:2407:x1871938519957248:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.910691:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.910698:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e3212baf/0xf62499ab9278f504 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1e9 expref: 218 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910708:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e3212baf/0xf62499ab9278f504 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1e9 expref: 218 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910721:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7293:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.910724:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7293:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.910729:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910737:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.910748:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.910777:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+218:2407:x1871938519957248:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (1086us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.910784:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100394 00010000:00010000:1.0:1785220581.910963:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.910977:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.910986:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.910994:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911018:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.911082:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510698496:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.911182:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911211:0:13819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e3212baf/0xf62499ab9278f504 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1e9 expref: 217 pid: 13866 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.911223:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510698240:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.911233:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911240:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911251:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911259:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911266:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911275:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.911312:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.911317:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911324:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.911327:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911335:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911346:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911349:0:14206:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ab3a22/0xf62499ab9278f440 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b8 expref: 216 pid: 13820 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911354:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000270ea293/0xf62499ab927a50cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.911360:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98405f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510695808:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.911368:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.911371:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911372:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911378:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.911381:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911381:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911388:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911390:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911397:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911399:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911405:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911406:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a511a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911413:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.911423:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.911426:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911432:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.911435:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911442:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911451:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911458:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a50db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911461:0:14203:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005259ddac/0xf62499ab9278ca9b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ddb4 expref: 260 pid: 14164 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911470:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.911470:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510697600:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.911472:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.911473:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.911481:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911488:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911495:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911503:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911510:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911527:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.911547:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.911551:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911558:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.911561:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911569:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911579:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911585:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a510c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.911597:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.911598:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.911600:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.911653:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98404000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1196:3655:x1871938510696704:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20012us (20180us total) trans 4295070047 rc 0/0 00000100:00100000:1.0:1785220581.911660:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284458 00000004:00080000:2.0:1785220581.911670:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98404700/2, rc 0, transno 4295070047 00000100:00100000:2.0:1785220581.911675:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98404700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696704:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.911694:0:14160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bdd42969/0xf62499ab9278c928 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dce2 expref: 259 pid: 13874 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.911704:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510697728:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.911715:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911722:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911728:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911735:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.911741:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.911806:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004966ee5a/0xf62499ab9278ce06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dee8 expref: 258 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.911835:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100395 00000100:00100000:0.0:1785220581.911845:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+258:2407:x1871938519957504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.911855:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.911860:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004966ee5a/0xf62499ab9278ce06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dee8 expref: 258 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.911869:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004966ee5a/0xf62499ab9278ce06 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dee8 expref: 258 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.911884:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004966ee5a/0xf62499ab9278ce06 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dee8 expref: 258 pid: 14171 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.911895:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7262:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.911898:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7262:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.911902:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.911910:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.911930:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.911955:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+257:2407:x1871938519957504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (2247us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.911961:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100395 00000100:00100000:0.0:1785220581.912060:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.912072:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510699776:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.912120:0:14202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005f778c18/0xf62499ab9278c768 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc02 expref: 256 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.912133:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.912173:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510699648:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.912196:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.912207:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510697472:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.912224:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.912231:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.912237:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.912245:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.912251:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.912274:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.912371:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.912557:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.912601:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.912606:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912614:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.912616:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912624:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912640:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912648:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5113 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.912655:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1196:3653:x1871938510696448:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20666us (20827us total) trans 4295070048 rc 0/0 00010000:00010000:2.0:1785220581.912662:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220581.912662:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284459 00010000:00010000:2.0:1785220581.912665:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912671:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.912673:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.912678:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510699776 00010000:00010000:2.0:1785220581.912681:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.912682:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510699776 00010000:00010000:2.0:1785220581.912690:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912696:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.912698:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.912700:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.912705:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510699648 00000100:00100000:1.0:1785220581.912708:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510699648 00000100:00100000:2.0:1785220581.912711:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284467 00000100:00100000:2.0:1785220581.912718:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1197:3653:x1871938510699776:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.912723:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284468 00000100:00100000:1.0:1785220581.912727:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1197:3655:x1871938510699648:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.912738:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29285 count 1 00002000:00080000:1.0:1785220581.912739:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29284 count 1 00010000:00010000:2.0:1785220581.912766:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.912766:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912780:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009d5b0dba/0xf62499ab9278d0de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e04d expref: 255 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.912780:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000098daae40/0xf62499ab9278d01a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfeb expref: 255 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912788:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7265:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.912788:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7264:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.912792:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7265:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.912792:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7264:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.912796:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.912797:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912822:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009d5b0dba/0xf62499ab9278d0de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e04d expref: 255 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.912822:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000098daae40/0xf62499ab9278d01a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfeb expref: 255 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.912833:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000098daae40/0xf62499ab9278d01a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfeb expref: 255 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.912843:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009d5b0dba/0xf62499ab9278d0de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e04d expref: 255 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.912845:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510699904:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.912852:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510700032:192.168.201.51@tcp:104: 00000004:00080000:0.0:1785220581.912908:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98404e00/2, rc 0, transno 4295070048 00000100:00100000:0.0:1785220581.912918:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98404e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510696448:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.912957:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510700160:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.913006:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd480 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.913025:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1785220581.913028:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.913030:0:14202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b41e0fba/0xf62499ab927a5105 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913037:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913045:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913051:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913059:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.913123:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519957760 00000100:00100000:1.0:1785220581.913128:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519957760 00010000:00010000:1.0:1785220581.913147:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000db928590/0xf62499ab9278f519 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f7 expref: 216 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.913166:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519957888 00000100:00100000:1.0:1785220581.913169:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519957888 00010000:00010000:1.0:1785220581.913181:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000630b2488/0xf62499ab9278f527 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f205 expref: 217 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.913199:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100396 00000100:00100000:1.0:1785220581.913206:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+217:2406:x1871938519957760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.913215:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.913220:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000db928590/0xf62499ab9278f519 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f7 expref: 217 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913236:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000db928590/0xf62499ab9278f519 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f7 expref: 217 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913249:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7294:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.913253:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7294:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.913257:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913270:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913283:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.913297:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+217:2406:x1871938519957760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (1030us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.913300:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.913304:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100396 00000100:00100000:1.0:1785220581.913316:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100397 00000100:00100000:1.0:1785220581.913322:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+216:2405:x1871938519957888:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.913329:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.913333:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000630b2488/0xf62499ab9278f527 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f205 expref: 216 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913342:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000630b2488/0xf62499ab9278f527 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f205 expref: 216 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913360:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7295:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.913363:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7295:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.913367:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913374:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913385:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.913402:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+216:2405:x1871938519957888:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 80us (1031us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.913409:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100397 00010000:00010000:1.0:1785220581.913430:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.913435:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913442:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.913446:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913453:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913465:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913477:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a5128 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7262:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913493:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.913495:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.913496:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.913503:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.913508:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.913532:0:14179:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000500f4f6e/0xf62499ab9278c9de lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd44 expref: 255 pid: 14163 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.913543:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98407480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510697088:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.913559:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913568:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913575:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913583:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913590:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.913614:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.913694:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98406680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1200:3652:x1871938510696576:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22689us (22738us total) trans 4295070051 rc 0/0 00000100:00100000:1.0:1785220581.913701:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284454 00010000:00010000:2.0:1785220581.913783:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.913788:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.913797:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.913800:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.913808:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.913820:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.913932:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a50f0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x725e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.913946:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.913948:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.913950:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.914211:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98407b80/2, rc 0, transno 4295070051 00000100:00100000:1.0:1785220581.914226:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98407b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510696576:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.914439:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.915779:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.915796:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.915807:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.915815:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.915841:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.916019:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1200:3652:x1871938510697984:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23461us (23633us total) trans 4295070052 rc 0/0 00000100:00100000:0.0:1785220581.916029:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284461 00000100:00100000:0.0:1785220581.916054:0:14260:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510700160 00000100:00100000:0.0:1785220581.916061:0:14260:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510700160 00000100:00100000:0.0:1785220581.916130:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284469 00000100:00100000:0.0:1785220581.916140:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1200:3653:x1871938510700160:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.916167:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29286 count 1 00010000:00010000:0.0:1785220581.916187:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916204:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e88d9247/0xf62499ab9278d204 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0d9 expref: 254 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916212:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7266:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.916216:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7266:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.916221:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916252:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e88d9247/0xf62499ab9278d204 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0d9 expref: 254 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916264:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e88d9247/0xf62499ab9278d204 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0d9 expref: 254 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916278:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510700288:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.916333:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.916367:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db928590/0xf62499ab9278f519 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f7 expref: 215 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916382:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510698880:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.916404:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916412:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916418:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916425:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916432:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916443:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220581.916511:0:14256:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000630b2488/0xf62499ab9278f527 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f205 expref: 214 pid: 14169 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916522:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510699008:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.916534:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916542:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916553:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916562:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916569:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916589:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220581.916607:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.916614:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916622:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.916626:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916631:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:2.0:1785220581.916635:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916641:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510700416:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.916651:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916664:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5121 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7261:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916672:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.916682:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.916685:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916692:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.916695:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916695:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510700416 00000100:00100000:0.0:1785220581.916698:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510700416 00010000:00010000:2.0:1785220581.916703:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916713:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916720:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284470 00010000:00010000:2.0:1785220581.916721:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a512f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.916725:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1201:3655:x1871938510700416:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.916734:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.916737:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220581.916738:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29287 count 1 00010000:00010000:2.0:1785220581.916744:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.916747:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916749:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916767:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916772:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007d11ae84/0xf62499ab9278d2e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e142 expref: 254 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916778:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916779:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7267:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.916783:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7267:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.916785:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5136 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916787:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.916805:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007d11ae84/0xf62499ab9278d2e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e142 expref: 254 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916814:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.916814:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007d11ae84/0xf62499ab9278d2e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e142 expref: 254 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.916816:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.916817:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.916824:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94913800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510700544:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.916856:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a90c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.917264:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1263:3655:x1871938510695296:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27075us (27491us total) trans 4295070634 rc 0/0 00000100:00100000:2.0:1785220581.917272:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284451 00000100:00100000:3.0:1785220581.917517:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1201:3654:x1871938510698368:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24288us (24465us total) trans 4295070050 rc 0/0 00000100:00100000:3.0:1785220581.917524:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284462 00000100:00100000:3.0:1785220581.917582:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510696064:12345-0@lo:6:osp-syn-0-0.0 Request processed in 26770us (26835us total) trans 4295070635 rc 0/0 00000100:00100000:3.0:1785220581.917589:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284453 00010000:00010000:3.0:1785220581.917605:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.917623:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98405180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1200:3655:x1871938510696192:12345-0@lo:6:osp-syn-1-0.0 Request processed in 26098us (26237us total) trans 4295070049 rc 0/0 00000100:00100000:3.0:1785220581.917628:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284457 00000100:00100000:3.0:1785220581.917705:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.917722:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958016 00000100:00100000:3.0:1785220581.917726:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958016 00010000:00010000:3.0:1785220581.917744:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000098daae40/0xf62499ab9278d01a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfeb expref: 255 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.917781:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100398 00000100:00100000:3.0:1785220581.917787:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+255:2407:x1871938519958016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.917797:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.917801:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000098daae40/0xf62499ab9278d01a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfeb expref: 255 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.917812:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000098daae40/0xf62499ab9278d01a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfeb expref: 255 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.917904:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1263:3655:x1871938510695168:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27680us (28090us total) trans 4295070636 rc 0/0 00000100:00100000:1.0:1785220581.917911:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284452 00000100:00100000:2.0:1785220581.917965:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510700032:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.917980:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.918001:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510699520:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.918009:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.918067:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1263:3654:x1871938510698624:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13574us (13690us total) trans 4295070637 rc 0/0 00000100:00100000:2.0:1785220581.918073:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284463 00000100:00100000:2.0:1785220581.918119:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.918208:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.918225:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.918277:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7264:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918281:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7264:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918287:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918296:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918311:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.918377:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+255:2407:x1871938519958016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 590us (673us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.918384:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100398 00000100:00100000:3.0:1785220581.918397:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958272 00000100:00100000:3.0:1785220581.918401:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958272 00010000:00010000:3.0:1785220581.918415:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ee1023ff/0xf62499ab9278f53c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f213 expref: 214 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.918434:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958144 00000100:00100000:3.0:1785220581.918437:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958144 00010000:00010000:3.0:1785220581.918447:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009d5b0dba/0xf62499ab9278d0de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e04d expref: 255 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.918465:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958400 00000100:00100000:1.0:1785220581.918465:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90e5c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+1263:3655:x1871938510694528:12345-0@lo:6:osp-syn-0-0.0 Request processed in 29213us (29466us total) trans 4295070638 rc 0/0 00000100:00100000:3.0:1785220581.918468:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958400 00000100:00100000:1.0:1785220581.918472:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284449 00010000:00010000:3.0:1785220581.918478:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cd2b1663/0xf62499ab9278cf4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df7b expref: 256 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.918494:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100399 00000100:00100000:3.0:1785220581.918499:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+214:2407:x1871938519958272:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.918506:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.918510:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ee1023ff/0xf62499ab9278f53c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f213 expref: 214 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918524:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ee1023ff/0xf62499ab9278f53c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f213 expref: 214 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918535:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7296:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918538:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7296:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918542:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918549:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918561:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.918579:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+214:2407:x1871938519958272:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.918586:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100399 00000100:00100000:1.0:1785220581.918591:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1263:3652:x1871938510697856:12345-0@lo:6:osp-syn-0-0.0 Request processed in 26137us (26347us total) trans 4295070639 rc 0/0 00000100:00100000:1.0:1785220581.918596:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284460 00000100:00100000:3.0:1785220581.918598:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100400 00000100:00100000:3.0:1785220581.918604:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+256:2407:x1871938519958144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.918611:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.918616:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009d5b0dba/0xf62499ab9278d0de lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e04d expref: 256 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918626:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009d5b0dba/0xf62499ab9278d0de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e04d expref: 256 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918644:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009d5b0dba/0xf62499ab9278d0de lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e04d expref: 256 pid: 14165 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.918645:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510698752:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13912us (14241us total) trans 4295070640 rc 0/0 00000100:00100000:1.0:1785220581.918651:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284464 00010000:00010000:3.0:1785220581.918656:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7265:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918658:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7265:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918662:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918670:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918678:0:14160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000098daae40/0xf62499ab9278d01a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfeb expref: 255 pid: 14160 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.918688:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510699904:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.918689:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.918699:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.918706:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+254:2407:x1871938519958144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (498us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.918712:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100400 00010000:00010000:1.0:1785220581.918714:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918720:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918728:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918736:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.918740:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100401 00000100:00100000:3.0:1785220581.918746:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+253:2407:x1871938519958400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.918750:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.918766:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.918771:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cd2b1663/0xf62499ab9278cf4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df7b expref: 253 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918779:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cd2b1663/0xf62499ab9278cf4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df7b expref: 253 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918796:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd2b1663/0xf62499ab9278cf4f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df7b expref: 253 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918806:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7263:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918809:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7263:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.918813:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918820:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.918848:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.918851:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1200:3655:x1871938510699648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6123us (6653us total) trans 4295070053 rc 0/0 00000100:00100000:1.0:1785220581.918857:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284468 00000100:00100000:3.0:1785220581.918864:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+252:2407:x1871938519958400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 118us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.918870:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100401 00010000:00010000:1.0:1785220581.918873:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.918878:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918886:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.918890:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918897:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918909:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918917:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a514b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7264:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.918932:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.918934:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.918936:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.918942:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.918947:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.919036:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee1023ff/0xf62499ab9278f53c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f213 expref: 213 pid: 14262 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.919052:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510699264:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.919066:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919074:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919080:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919108:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919116:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919130:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.919208:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510699136:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10538us (10624us total) trans 4295070641 rc 0/0 00000100:00100000:0.0:1785220581.919214:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284465 00010000:00010000:0.0:1785220581.919316:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.919321:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919328:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.919331:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919338:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919348:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919354:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a513d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.919376:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.919379:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.919380:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.919405:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0f800/2, rc 0, transno 4295070641 00000100:00100000:0.0:1785220581.919417:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510699136:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220581.919505:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919515:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919523:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919531:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919550:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1785220581.919575:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0d880/2, rc 0, transno 4295070052 00000100:00100000:1.0:1785220581.919586:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510697984:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.919620:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1785220581.919625:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9363aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1200:3653:x1871938510699776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6908us (7492us total) trans 4295070054 rc 0/0 00010000:00010000:1.0:1785220581.919625:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919632:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:2.0:1785220581.919633:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284467 00010000:00010000:1.0:1785220581.919635:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919643:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919649:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919654:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919664:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919664:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5152 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7265:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919672:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.919678:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919678:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.919679:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.919681:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.919690:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.919706:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.919710:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919717:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.919720:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919728:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919738:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.919746:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a5144 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7263:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.919769:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1200:3655:x1871938510699392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10235us (10437us total) trans 4295070055 rc 0/0 00010000:00010000:1.0:1785220581.919769:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.919771:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.919773:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220581.919776:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284466 00000004:00080000:0.0:1785220581.919807:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0e300/2, rc 0, transno 4295070640 00000100:00100000:0.0:1785220581.919825:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510698752:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.919844:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0d500/2, rc 0, transno 4295070639 00000100:00100000:1.0:1785220581.919848:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510697856:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.919862:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510700800:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.920037:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.920070:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220581.920076:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510700800 00000100:00100000:1.0:1785220581.920076:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510700928:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.920080:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510700800 00000100:00100000:2.0:1785220581.920122:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284471 00000100:00100000:2.0:1785220581.920199:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1200:3652:x1871938510700800:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.920219:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29288 count 1 00010000:00010000:2.0:1785220581.920238:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.920254:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bb171cc1/0xf62499ab9278d426 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e1e3 expref: 251 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.920261:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7268:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.920265:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7268:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.920269:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.920292:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bb171cc1/0xf62499ab9278d426 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e1e3 expref: 251 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.920303:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bb171cc1/0xf62499ab9278d426 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e1e3 expref: 251 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.920314:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510701184:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.920356:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000004:00080000:0.0:1785220581.921179:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0ea00/2, rc 0, transno 4295070635 00000100:00100000:0.0:1785220581.921194:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510696064:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220581.921209:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0fb80/2, rc 0, transno 4295070054 00000100:00100000:0.0:1785220581.921221:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510699776:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.921232:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510700672:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.921261:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:1.0:1785220581.921264:0:8435:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4295070633 is committed 00000100:00100000:2.0:1785220581.921287:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510701312:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.921305:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.921324:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510700672 00000100:00100000:2.0:1785220581.921328:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510700672 00000100:00100000:2.0:1785220581.921351:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510701312 00000100:00100000:2.0:1785220581.921353:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510701312 00000100:00100000:2.0:1785220581.921367:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284472 00000100:00100000:2.0:1785220581.921373:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1196:3655:x1871938510700672:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.921386:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29335 count 1 00010000:00010000:2.0:1785220581.921400:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921413:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cee0eaae/0xf62499ab9278f5eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f221 expref: 212 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921421:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7297:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220581.921424:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7297:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.921427:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921444:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cee0eaae/0xf62499ab9278f5eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f221 expref: 212 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921453:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cee0eaae/0xf62499ab9278f5eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f221 expref: 212 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.921463:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510701440:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.921497:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.921511:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284473 00000100:00100000:2.0:1785220581.921517:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1079:3655:x1871938510701312:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.921527:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29338 count 1 00010000:00010000:2.0:1785220581.921536:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921547:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eed7e968/0xf62499ab9278f788 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f24b expref: 212 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921553:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x729a:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220581.921556:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.921560:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921574:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eed7e968/0xf62499ab9278f788 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f24b expref: 212 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.921582:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eed7e968/0xf62499ab9278f788 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f24b expref: 212 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.921591:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510701568:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.921609:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df010c0 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.921632:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0f480/2, rc 0, transno 4295070053 00000100:00100000:2.0:1785220581.921636:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510699648:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.921644:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0d180/2, rc 0, transno 4295070055 00000100:00100000:2.0:1785220581.921647:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510699392:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.921655:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98407800/2, rc 0, transno 4295070049 00000100:00100000:2.0:1785220581.921658:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98407800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510696192:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.921667:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89eff100/2, rc 0, transno 4295070636 00000100:00100000:2.0:1785220581.921673:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510695168:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.921702:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efd880/2, rc 0, transno 4295070634 00000100:00100000:2.0:1785220581.921707:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510695296:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220581.921728:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efed80/2, rc 0, transno 4295070638 00000100:00100000:2.0:1785220581.921733:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510694528:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.921752:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1785220581.921771:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510701056:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.921794:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.921794:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510701824:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.921814:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510701056 00000100:00100000:2.0:1785220581.921818:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510701056 00000100:00100000:0.0:1785220581.921818:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.921838:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.921843:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510701696:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.921865:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.922329:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510701824 00000100:00100000:2.0:1785220581.922334:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510701824 00000100:00100000:0.0:1785220581.922340:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510702336:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.922353:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510701696 00000100:00100000:2.0:1785220581.922355:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510701696 00000100:00100000:0.0:1785220581.922366:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.922389:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220581.922395:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510702208:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.922417:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:1.0:1785220581.922583:0:8435:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4295069983 is committed 00000100:00100000:2.0:1785220581.922773:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510702336 00000100:00100000:2.0:1785220581.922778:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510702336 00000100:00100000:2.0:1785220581.922806:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284474 00000100:00100000:2.0:1785220581.922813:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+557:3655:x1871938510701056:12345-0@lo:6:osp-syn-0-0.0 00000001:00080000:1.0:1785220581.923307:0:8435:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4295069807 is committed 00002000:00080000:2.0:1785220581.923494:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29337 count 1 00010000:00010000:2.0:1785220581.923513:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923530:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009a4dd1e5/0xf62499ab9278f703 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f23d expref: 212 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923537:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7299:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220581.923541:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7299:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.923545:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923568:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009a4dd1e5/0xf62499ab9278f703 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f23d expref: 212 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923579:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009a4dd1e5/0xf62499ab9278f703 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f23d expref: 212 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.923591:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510702592:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.923634:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.923655:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284475 00000100:00100000:2.0:1785220581.923661:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+388:3653:x1871938510701824:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.923672:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29340 count 1 00010000:00010000:2.0:1785220581.923682:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923693:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000882a968d/0xf62499ab9278f8c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f267 expref: 212 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923699:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x729c:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220581.923702:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.923705:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923721:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000882a968d/0xf62499ab9278f8c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f267 expref: 212 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.923729:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000882a968d/0xf62499ab9278f8c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f267 expref: 212 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.923738:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510702720:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.923774:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e82bfd480 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.923790:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510701952:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.923817:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.924049:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510702080:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.924071:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.925015:0:14261:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510701952 00000100:00100000:2.0:1785220581.925024:0:14261:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510701952 00000100:00100000:0.0:1785220581.925260:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510702080 00000100:00100000:0.0:1785220581.925266:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510702080 00000100:00100000:0.0:1785220581.925294:0:14150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510702208 00000100:00100000:0.0:1785220581.925297:0:14150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510702208 00000100:00100000:0.0:1785220581.925318:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284476 00000100:00100000:0.0:1785220581.925326:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+33:3653:x1871938510701696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.925346:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29339 count 1 00010000:00010000:0.0:1785220581.925366:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.925370:0:14261:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284477 00000100:00100000:2.0:1785220581.925378:0:14261:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1203:3653:x1871938510702336:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.925382:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b087cbec/0xf62499ab9278f829 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f259 expref: 212 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.925390:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x729b:0x0].0x0 since=20s v=(0 0) 00002000:00080000:2.0:1785220581.925391:0:14261:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29291 count 1 00010000:00010000:0.0:1785220581.925394:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.925397:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925404:0:14261:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925414:0:14261:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000058a41ff9/0xf62499ab9278d767 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e341 expref: 251 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925420:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x726b:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.925421:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b087cbec/0xf62499ab9278f829 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f259 expref: 212 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925423:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.925427:0:14261:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.925433:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b087cbec/0xf62499ab9278f829 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f259 expref: 212 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925443:0:14261:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000058a41ff9/0xf62499ab9278d767 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e341 expref: 251 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.925445:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510702848:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.925452:0:14261:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000058a41ff9/0xf62499ab9278d767 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e341 expref: 251 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.925467:0:14261:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510702976:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.925489:0:14261:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.925489:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9900 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.925507:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284478 00000100:00100000:0.0:1785220581.925585:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+33:3655:x1871938510701952:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.925601:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29341 count 1 00010000:00010000:0.0:1785220581.925613:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.925625:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284479 00010000:00010000:0.0:1785220581.925629:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000015ca6e7b/0xf62499ab9278f93a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f275 expref: 212 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.925633:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1203:3655:x1871938510702080:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.925636:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x729d:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220581.925639:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.925643:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220581.925647:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29289 count 1 00010000:00010000:2.0:1785220581.925658:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.925660:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000015ca6e7b/0xf62499ab9278f93a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f275 expref: 212 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.925670:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000015ca6e7b/0xf62499ab9278f93a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f275 expref: 212 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925671:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008f9711bc/0xf62499ab9278d4ff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e222 expref: 251 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925679:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7269:0x0].0x0 since=24s v=(0 0) 00000100:00100000:0.0:1785220581.925681:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510703104:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.925682:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7269:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.925686:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.925703:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.925705:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008f9711bc/0xf62499ab9278d4ff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e222 expref: 251 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.925714:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008f9711bc/0xf62499ab9278d4ff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e222 expref: 251 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.925718:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284480 00000100:00100000:2.0:1785220581.925724:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510703232:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.925725:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d985fe450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1203:3653:x1871938510702208:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.925736:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29290 count 1 00000100:00100000:2.0:1785220581.925747:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.925747:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.925776:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000588f8e63/0xf62499ab9278d62c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e2a7 expref: 251 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.925777:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.925783:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510702464:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.925783:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x726a:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.925786:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.925789:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.925807:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000588f8e63/0xf62499ab9278d62c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e2a7 expref: 251 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.925813:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.925816:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000588f8e63/0xf62499ab9278d62c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e2a7 expref: 251 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.925826:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510703360:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.925941:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510702464 00000100:00100000:1.0:1785220581.925945:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510702464 00000100:00100000:3.0:1785220581.925987:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284481 00000100:00100000:0.0:1785220581.925988:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.925998:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1204:3655:x1871938510702464:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.926010:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29292 count 1 00010000:00010000:3.0:1785220581.926021:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.926033:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000065bfbd4c/0xf62499ab9278d919 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e3d4 expref: 251 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.926040:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x726c:0x0].0x0 since=23s v=(0 0) 00010000:00010000:3.0:1785220581.926043:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.926047:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.926064:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000065bfbd4c/0xf62499ab9278d919 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e3d4 expref: 251 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.926071:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220581.926073:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000065bfbd4c/0xf62499ab9278d919 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e3d4 expref: 251 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.926083:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510703488:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.926118:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f240 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.926254:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0ed80/2, rc 0, transno 4295070637 00000100:00100000:2.0:1785220581.926265:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510698624:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.926315:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510703616:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.926336:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.926414:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0f100/2, rc 0, transno 4295070050 00000100:00100000:2.0:1785220581.926429:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510698368:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.926570:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.926604:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.926660:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510701184:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.926680:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.927082:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.931939:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510700928 00000100:00100000:0.0:1785220581.931939:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510703616 00000100:00100000:3.0:1785220581.931952:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510700928 00000100:00100000:0.0:1785220581.931952:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510703616 00000100:00100000:0.0:1785220581.932013:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284482 00000100:00100000:0.0:1785220581.932023:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+20:3655:x1871938510703616:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.932053:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29342 count 1 00010000:00010000:0.0:1785220581.932078:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932241:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007b5aac2b/0xf62499ab9278fa0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f283 expref: 212 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932251:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x729e:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220581.932256:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.932261:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932299:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007b5aac2b/0xf62499ab9278fa0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f283 expref: 212 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932312:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007b5aac2b/0xf62499ab9278fa0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f283 expref: 212 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.932327:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0000_UUID:14145:1871938510703744:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.932409:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.932511:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284483 00000100:00100000:0.0:1785220581.932520:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+20:3652:x1871938510700928:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.932536:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29336 count 1 00010000:00010000:0.0:1785220581.932547:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932559:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8afc1a7/0xf62499ab9278f67e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f22f expref: 212 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932565:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7298:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220581.932568:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7298:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.932571:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932588:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b8afc1a7/0xf62499ab9278f67e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f22f expref: 212 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.932593:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510701440:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.932598:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b8afc1a7/0xf62499ab9278f67e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f22f expref: 212 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.932607:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510703872:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.932615:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.932632:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.932655:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510700288:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.932669:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.932691:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510701568:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.932692:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94913800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510700544:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.932703:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.932703:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.932783:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958528 00000100:00100000:1.0:1785220581.932788:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958528 00010000:00010000:1.0:1785220581.932814:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e88d9247/0xf62499ab9278d204 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0d9 expref: 252 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.932877:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958656 00000100:00100000:1.0:1785220581.932881:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958656 00010000:00010000:1.0:1785220581.932894:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007d11ae84/0xf62499ab9278d2e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e142 expref: 253 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.932922:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958784 00000100:00100000:0.0:1785220581.932925:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958784 00010000:00010000:0.0:1785220581.932941:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bb171cc1/0xf62499ab9278d426 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e1e3 expref: 254 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.932965:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100402 00000100:00100000:0.0:1785220581.932973:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+254:2407:x1871938519958528:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.932983:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.932989:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e88d9247/0xf62499ab9278d204 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0d9 expref: 254 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.933000:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e88d9247/0xf62499ab9278d204 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0d9 expref: 254 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.933024:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e88d9247/0xf62499ab9278d204 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e0d9 expref: 254 pid: 14207 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.933037:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7266:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.933041:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7266:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.933046:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.933055:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.933073:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.933179:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510702592:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.933193:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.933251:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.933274:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.933296:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.933317:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.933332:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959040 00000100:00100000:2.0:1785220581.933334:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.933336:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959040 00010000:00010000:1.0:1785220581.933351:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eed7e968/0xf62499ab9278f788 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f24b expref: 213 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.933367:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510702976:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.933374:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959296 00000100:00100000:1.0:1785220581.933376:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959296 00010000:00010000:2.0:1785220581.933380:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933387:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000882a968d/0xf62499ab9278f8c3 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f267 expref: 214 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933403:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519958912 00000100:00100000:1.0:1785220581.933405:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519958912 00010000:00010000:1.0:1785220581.933416:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cee0eaae/0xf62499ab9278f5eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f221 expref: 215 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.933432:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510703232:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.933432:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959168 00000100:00100000:1.0:1785220581.933434:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959168 00010000:00010000:2.0:1785220581.933445:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933445:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009a4dd1e5/0xf62499ab9278f703 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f23d expref: 216 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933461:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959424 00000100:00100000:1.0:1785220581.933463:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959424 00010000:00010000:1.0:1785220581.933474:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b087cbec/0xf62499ab9278f829 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f259 expref: 217 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933488:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100403 00000100:00100000:1.0:1785220581.933495:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+253:2407:x1871938519958656:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220581.933498:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.933503:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.933507:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007d11ae84/0xf62499ab9278d2e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e142 expref: 253 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.933509:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.933521:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.933523:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007d11ae84/0xf62499ab9278d2e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e142 expref: 253 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.933531:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.933537:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007d11ae84/0xf62499ab9278d2e4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e142 expref: 253 pid: 14248 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.933547:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.933548:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7267:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.933551:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7267:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.933555:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933562:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.933566:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510702720:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.933577:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933578:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.933607:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+252:2407:x1871938519958656:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (7002us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.933615:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100403 00000100:00100000:1.0:1785220581.933631:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959680 00000100:00100000:1.0:1785220581.933634:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959680 00010000:00010000:1.0:1785220581.933646:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000015ca6e7b/0xf62499ab9278f93a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f275 expref: 218 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933659:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959936 00000100:00100000:1.0:1785220581.933661:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959936 00010000:00010000:1.0:1785220581.933674:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000588f8e63/0xf62499ab9278d62c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e2a7 expref: 252 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933685:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959552 00000100:00100000:1.0:1785220581.933687:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959552 00010000:00010000:1.0:1785220581.933698:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000058a41ff9/0xf62499ab9278d767 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e341 expref: 253 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933707:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519959808 00000100:00100000:1.0:1785220581.933710:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519959808 00010000:00010000:1.0:1785220581.933719:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008f9711bc/0xf62499ab9278d4ff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e222 expref: 254 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933729:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960064 00000100:00100000:1.0:1785220581.933731:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960064 00010000:00010000:1.0:1785220581.933742:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000065bfbd4c/0xf62499ab9278d919 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e3d4 expref: 255 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.933765:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100404 00000100:00100000:1.0:1785220581.933773:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+255:2407:x1871938519958784:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.933781:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.933785:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bb171cc1/0xf62499ab9278d426 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e1e3 expref: 255 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933793:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bb171cc1/0xf62499ab9278d426 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e1e3 expref: 255 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933811:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb171cc1/0xf62499ab9278d426 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e1e3 expref: 255 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933820:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7268:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.933823:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7268:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.933828:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933835:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933859:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.933886:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+254:2407:x1871938519958784:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (6805us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.933894:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100404 00000100:00100000:1.0:1785220581.933908:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100405 00000100:00100000:1.0:1785220581.933915:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+218:2407:x1871938519959040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.933923:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.933928:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eed7e968/0xf62499ab9278f788 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f24b expref: 218 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933939:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eed7e968/0xf62499ab9278f788 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f24b expref: 218 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.933958:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933959:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eed7e968/0xf62499ab9278f788 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f24b expref: 218 pid: 14192 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.933968:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933970:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.933973:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.933977:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.933979:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.933985:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.933987:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.934001:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.934001:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.934019:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+217:2407:x1871938519959040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (769us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.934024:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100405 00000100:00100000:1.0:1785220581.934037:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100406 00000100:00100000:1.0:1785220581.934041:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+216:2407:x1871938519959296:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.934048:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.934052:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000882a968d/0xf62499ab9278f8c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f267 expref: 216 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.934061:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000882a968d/0xf62499ab9278f8c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f267 expref: 216 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.934076:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000882a968d/0xf62499ab9278f8c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f267 expref: 216 pid: 14172 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.934176:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1205:3652:x1871938510700800:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13978us (14138us total) trans 4295070056 rc 0/0 00000100:00100000:2.0:1785220581.934184:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284471 00010000:00010000:2.0:1785220581.934205:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.934214:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.934222:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.934230:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.934244:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.934277:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+253:2407:x1871938519958528:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1304us (7710us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.934285:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100402 00000100:00100000:0.0:1785220581.934300:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100407 00000100:00100000:0.0:1785220581.934306:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e13050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+215:2407:x1871938519958912:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.934316:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.934321:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cee0eaae/0xf62499ab9278f5eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f221 expref: 215 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.934330:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cee0eaae/0xf62499ab9278f5eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f221 expref: 215 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.934338:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+21:3655:x1871938510701312:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12821us (13031us total) trans 4295070642 rc 0/0 00010000:00010000:0.0:1785220581.934343:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cee0eaae/0xf62499ab9278f5eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f221 expref: 215 pid: 14156 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.934344:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284473 00010000:00010000:0.0:1785220581.934353:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7297:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.934356:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7297:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.934360:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.934366:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.934375:0:3655:0:(client.c:3112:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4295070633 gen 2 00000004:00080000:2.0:1785220581.934380:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c972d80, transno 4295068787 00010000:00010000:0.0:1785220581.934384:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.934388:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af8a80, transno 4295068788 00000004:00080000:2.0:1785220581.934390:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91171880, transno 4295068789 00000004:00080000:2.0:1785220581.934393:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be0a80, transno 4295068790 00000004:00080000:2.0:1785220581.934395:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364aa00, transno 4295068794 00000004:00080000:2.0:1785220581.934398:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480ca80, transno 4295068795 00000004:00080000:2.0:1785220581.934401:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1bb80, transno 4295068799 00000100:00100000:0.0:1785220581.934401:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e13050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+214:2407:x1871938519958912:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (1104us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.934403:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c18700, transno 4295068800 00000004:00080000:2.0:1785220581.934406:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7ca80, transno 4295068803 00000100:00100000:0.0:1785220581.934407:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100407 00000004:00080000:2.0:1785220581.934408:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7ea00, transno 4295068805 00000004:00080000:2.0:1785220581.934411:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91583100, transno 4295068806 00000004:00080000:2.0:1785220581.934413:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91582a00, transno 4295068807 00000004:00080000:2.0:1785220581.934415:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91581f80, transno 4295068809 00000004:00080000:2.0:1785220581.934418:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb3100, transno 4295068811 00000100:00100000:0.0:1785220581.934419:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100408 00000004:00080000:2.0:1785220581.934421:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95451f80, transno 4295068813 00000004:00080000:2.0:1785220581.934423:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e31f80, transno 4295068816 00000100:00100000:0.0:1785220581.934424:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+213:2407:x1871938519959168:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000004:00080000:2.0:1785220581.934426:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e31500, transno 4295068819 00000004:00080000:2.0:1785220581.934429:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb85180, transno 4295068820 00010000:00010000:0.0:1785220581.934431:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:2.0:1785220581.934432:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb84000, transno 4295068822 00000004:00080000:2.0:1785220581.934434:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af9500, transno 4295068823 00010000:00010000:0.0:1785220581.934436:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009a4dd1e5/0xf62499ab9278f703 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f23d expref: 213 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934437:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93628700, transno 4295068824 00000004:00080000:2.0:1785220581.934440:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93629180, transno 4295068825 00000004:00080000:2.0:1785220581.934443:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b47100, transno 4295068826 00000004:00080000:2.0:1785220581.934445:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b45880, transno 4295068827 00010000:00010000:0.0:1785220581.934446:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009a4dd1e5/0xf62499ab9278f703 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f23d expref: 213 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934448:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adad80, transno 4295068829 00000004:00080000:2.0:1785220581.934451:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad9180, transno 4295068831 00000004:00080000:2.0:1785220581.934453:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad9c00, transno 4295068834 00000004:00080000:2.0:1785220581.934455:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adaa00, transno 4295068835 00000004:00080000:2.0:1785220581.934458:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b44a80, transno 4295068836 00010000:00010000:0.0:1785220581.934458:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a4dd1e5/0xf62499ab9278f703 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f23d expref: 213 pid: 14209 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934461:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad8e00, transno 4295068838 00000004:00080000:2.0:1785220581.934463:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bca80, transno 4295068842 00000004:00080000:2.0:1785220581.934466:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce9f80, transno 4295068843 00010000:00010000:0.0:1785220581.934467:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7299:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934468:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cea300, transno 4295068845 00010000:00010000:0.0:1785220581.934470:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7299:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934471:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95450380, transno 4295068846 00000004:00080000:2.0:1785220581.934473:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c169f80, transno 4295068848 00010000:00010000:0.0:1785220581.934474:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934476:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16b100, transno 4295068850 00000004:00080000:2.0:1785220581.934478:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16ad80, transno 4295068852 00000004:00080000:2.0:1785220581.934480:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af6680, transno 4295068853 00010000:00010000:0.0:1785220581.934482:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934483:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90407b80, transno 4295068856 00000004:00080000:2.0:1785220581.934486:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95539c00, transno 4295068858 00000004:00080000:2.0:1785220581.934488:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95452680, transno 4295068860 00000004:00080000:2.0:1785220581.934490:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95452d80, transno 4295068861 00000004:00080000:2.0:1785220581.934492:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e31880, transno 4295068863 00000004:00080000:2.0:1785220581.934495:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbd880, transno 4295068865 00010000:00010000:0.0:1785220581.934497:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.934498:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91583480, transno 4295068866 00000004:00080000:2.0:1785220581.934500:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91580700, transno 4295068869 00000004:00080000:2.0:1785220581.934503:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93628380, transno 4295068871 00000004:00080000:2.0:1785220581.934505:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98485500, transno 4295068874 00000004:00080000:2.0:1785220581.934508:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98625180, transno 4295068877 00000004:00080000:2.0:1785220581.934511:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b6680, transno 4295068878 00000100:00100000:0.0:1785220581.934511:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+212:2407:x1871938519959168:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 87us (1194us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.934513:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98625880, transno 4295068880 00000004:00080000:2.0:1785220581.934516:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d85ee2a00, transno 4295068881 00000100:00100000:0.0:1785220581.934517:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100408 00000004:00080000:2.0:1785220581.934518:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e33b80, transno 4295068884 00000004:00080000:2.0:1785220581.934521:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91583800, transno 4295068885 00000004:00080000:2.0:1785220581.934523:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91580e00, transno 4295068886 00000004:00080000:2.0:1785220581.934526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e32300, transno 4295068888 00000100:00100000:0.0:1785220581.934527:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100409 00000004:00080000:2.0:1785220581.934529:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e30000, transno 4295068893 00000004:00080000:2.0:1785220581.934531:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e32680, transno 4295068894 00000100:00100000:0.0:1785220581.934532:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+211:2407:x1871938519959424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000004:00080000:2.0:1785220581.934534:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce9180, transno 4295068897 00000004:00080000:2.0:1785220581.934536:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce9500, transno 4295068898 00000004:00080000:2.0:1785220581.934539:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95422d80, transno 4295068903 00010000:00010000:0.0:1785220581.934539:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:2.0:1785220581.934542:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f68a80, transno 4295068904 00010000:00010000:0.0:1785220581.934543:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b087cbec/0xf62499ab9278f829 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f259 expref: 211 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934544:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98485c00, transno 4295068906 00000004:00080000:2.0:1785220581.934547:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95422a00, transno 4295068907 00000004:00080000:2.0:1785220581.934550:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d792680, transno 4295068908 00000004:00080000:2.0:1785220581.934552:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d790700, transno 4295068910 00000004:00080000:2.0:1785220581.934555:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984c000, transno 4295068913 00000004:00080000:2.0:1785220581.934558:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984c380, transno 4295068914 00010000:00010000:0.0:1785220581.934558:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b087cbec/0xf62499ab9278f829 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f259 expref: 211 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934560:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93631c00, transno 4295068921 00000004:00080000:2.0:1785220581.934563:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93633480, transno 4295068922 00000004:00080000:2.0:1785220581.934565:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce8a80, transno 4295068926 00000004:00080000:2.0:1785220581.934568:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986d880, transno 4295068927 00010000:00010000:0.0:1785220581.934569:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729b:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934571:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93630000, transno 4295068928 00010000:00010000:0.0:1785220581.934572:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729b:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934573:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986d500, transno 4295068929 00010000:00010000:0.0:1785220581.934575:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934576:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93631500, transno 4295068933 00000004:00080000:2.0:1785220581.934578:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce8000, transno 4295068935 00000004:00080000:2.0:1785220581.934580:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6b480, transno 4295068936 00000004:00080000:2.0:1785220581.934583:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca8e00, transno 4295068941 00010000:00010000:0.0:1785220581.934583:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934585:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d978fea00, transno 4295068942 00000004:00080000:2.0:1785220581.934589:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c63100, transno 4295068943 00000004:00080000:2.0:1785220581.934591:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c62680, transno 4295068944 00010000:00010000:0.0:1785220581.934592:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.934594:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bf800, transno 4295068947 00000004:00080000:2.0:1785220581.934596:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcab480, transno 4295068948 00000004:00080000:2.0:1785220581.934599:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979eaa00, transno 4295068949 00000004:00080000:2.0:1785220581.934601:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979eb800, transno 4295068951 00000100:00100000:0.0:1785220581.934604:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+211:2407:x1871938519959424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 73us (1270us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.934605:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a9180, transno 4295068955 00000004:00080000:2.0:1785220581.934607:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b9f80, transno 4295068957 00000004:00080000:2.0:1785220581.934610:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979ba300, transno 4295068958 00000100:00100000:0.0:1785220581.934611:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100409 00000004:00080000:2.0:1785220581.934612:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b9c00, transno 4295068959 00000004:00080000:2.0:1785220581.934615:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979bbb80, transno 4295068960 00000004:00080000:2.0:1785220581.934617:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e8700, transno 4295068965 00000004:00080000:2.0:1785220581.934620:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e9180, transno 4295068969 00000100:00100000:0.0:1785220581.934621:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100410 00000004:00080000:2.0:1785220581.934622:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bca80, transno 4295068970 00000004:00080000:2.0:1785220581.934625:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996ed80, transno 4295068971 00000100:00100000:0.0:1785220581.934626:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+210:2406:x1871938519959680:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000004:00080000:2.0:1785220581.934628:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b8000, transno 4295068972 00000004:00080000:2.0:1785220581.934631:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19d180, transno 4295068975 00000004:00080000:2.0:1785220581.934633:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996c000, transno 4295068976 00010000:00010000:0.0:1785220581.934634:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:2.0:1785220581.934636:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19c380, transno 4295068977 00010000:00010000:0.0:1785220581.934638:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000015ca6e7b/0xf62499ab9278f93a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f275 expref: 210 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934639:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f69180, transno 4295068981 00000004:00080000:2.0:1785220581.934642:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236b100, transno 4295068983 00000004:00080000:2.0:1785220581.934644:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92369880, transno 4295068985 00000004:00080000:2.0:1785220581.934647:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19d500, transno 4295068986 00010000:00010000:0.0:1785220581.934648:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000015ca6e7b/0xf62499ab9278f93a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f275 expref: 210 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934649:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92368e00, transno 4295068987 00000004:00080000:2.0:1785220581.934652:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236a300, transno 4295068988 00000004:00080000:2.0:1785220581.934655:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996d180, transno 4295068992 00000004:00080000:2.0:1785220581.934658:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5dc00, transno 4295068993 00000004:00080000:2.0:1785220581.934660:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d978fe680, transno 4295068998 00000004:00080000:2.0:1785220581.934663:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995f100, transno 4295069000 00010000:00010000:0.0:1785220581.934664:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729d:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934665:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97954700, transno 4295069002 00010000:00010000:0.0:1785220581.934667:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729d:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934668:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19f800, transno 4295069007 00000004:00080000:2.0:1785220581.934670:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97955c00, transno 4295069008 00010000:00010000:0.0:1785220581.934671:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934673:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97956300, transno 4295069009 00000004:00080000:2.0:1785220581.934675:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19df80, transno 4295069010 00000004:00080000:2.0:1785220581.934678:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e9c00, transno 4295069013 00010000:00010000:0.0:1785220581.934679:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995df80, transno 4295069014 00000004:00080000:2.0:1785220581.934683:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19e680, transno 4295069017 00000004:00080000:2.0:1785220581.934685:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97954a80, transno 4295069018 00000004:00080000:2.0:1785220581.934688:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d4700, transno 4295069021 00010000:00010000:0.0:1785220581.934688:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.934690:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d6d80, transno 4295069022 00000004:00080000:2.0:1785220581.934693:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d4a80, transno 4295069024 00000004:00080000:2.0:1785220581.934695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94deea00, transno 4295069026 00000004:00080000:2.0:1785220581.934698:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ded500, transno 4295069029 00000004:00080000:2.0:1785220581.934700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94def800, transno 4295069032 00000100:00100000:0.0:1785220581.934701:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+210:2406:x1871938519959680:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (1203us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.934703:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dee300, transno 4295069033 00000004:00080000:2.0:1785220581.934706:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92369180, transno 4295069034 00000100:00100000:0.0:1785220581.934706:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100410 00000004:00080000:2.0:1785220581.934708:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98410000, transno 4295069035 00000004:00080000:2.0:1785220581.934711:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236ad80, transno 4295069037 00000004:00080000:2.0:1785220581.934713:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98410e00, transno 4295069039 00000004:00080000:2.0:1785220581.934716:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c60700, transno 4295069041 00000100:00100000:0.0:1785220581.934716:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100411 00000004:00080000:2.0:1785220581.934719:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00e300, transno 4295069045 00000100:00100000:0.0:1785220581.934721:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+252:2406:x1871938519959936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000004:00080000:2.0:1785220581.934722:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f0700, transno 4295069046 00000004:00080000:2.0:1785220581.934724:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684f480, transno 4295069049 00000004:00080000:2.0:1785220581.934727:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6fb80, transno 4295069050 00010000:00010000:0.0:1785220581.934728:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:2.0:1785220581.934729:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6dc00, transno 4295069051 00000004:00080000:2.0:1785220581.934732:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6d880, transno 4295069054 00010000:00010000:0.0:1785220581.934733:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000588f8e63/0xf62499ab9278d62c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e2a7 expref: 252 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934734:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97441880, transno 4295069056 00000004:00080000:2.0:1785220581.934737:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97441f80, transno 4295069058 00000004:00080000:2.0:1785220581.934739:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d5f80, transno 4295069061 00010000:00010000:0.0:1785220581.934741:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000588f8e63/0xf62499ab9278d62c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e2a7 expref: 252 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934742:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bc000, transno 4295069065 00000004:00080000:2.0:1785220581.934745:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bd880, transno 4295069067 00000004:00080000:2.0:1785220581.934747:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bea00, transno 4295069069 00000004:00080000:2.0:1785220581.934750:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8ed80, transno 4295069071 00000004:00080000:2.0:1785220581.934753:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f0380, transno 4295069072 00010000:00010000:0.0:1785220581.934753:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726a:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d7100, transno 4295069073 00010000:00010000:0.0:1785220581.934768:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726a:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.934772:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfeb100, transno 4295069074 00010000:00010000:0.0:1785220581.934773:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934775:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735df80, transno 4295069079 00000004:00080000:2.0:1785220581.934777:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735c000, transno 4295069081 00000004:00080000:2.0:1785220581.934780:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735fb80, transno 4295069082 00010000:00010000:0.0:1785220581.934781:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934783:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735f800, transno 4295069084 00000004:00080000:2.0:1785220581.934786:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f55500, transno 4295069085 00000004:00080000:2.0:1785220581.934788:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ef100, transno 4295069087 00000004:00080000:2.0:1785220581.934791:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f55f80, transno 4295069088 00010000:00010000:0.0:1785220581.934792:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.934793:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ee300, transno 4295069090 00000004:00080000:2.0:1785220581.934796:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae20000, transno 4295069097 00000004:00080000:2.0:1785220581.934798:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979eb100, transno 4295069099 00000004:00080000:2.0:1785220581.934800:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae20e00, transno 4295069102 00000004:00080000:2.0:1785220581.934802:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae22d80, transno 4295069104 00000004:00080000:2.0:1785220581.934805:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae23480, transno 4295069105 00000100:00100000:0.0:1785220581.934807:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+252:2406:x1871938519959936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (1297us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.934808:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae22680, transno 4295069106 00000004:00080000:2.0:1785220581.934811:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977ab800, transno 4295069107 00000100:00100000:0.0:1785220581.934813:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100411 00000004:00080000:2.0:1785220581.934814:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995fb80, transno 4295069108 00000004:00080000:2.0:1785220581.934816:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995ca80, transno 4295069113 00000004:00080000:2.0:1785220581.934819:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad6680, transno 4295069115 00000004:00080000:2.0:1785220581.934822:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbc700, transno 4295069116 00000100:00100000:0.0:1785220581.934823:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100412 00000100:00100000:0.0:1785220581.934829:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+251:2406:x1871938519959552:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.934859:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.934865:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000058a41ff9/0xf62499ab9278d767 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e341 expref: 251 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.934874:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000058a41ff9/0xf62499ab9278d767 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e341 expref: 251 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.934887:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000058a41ff9/0xf62499ab9278d767 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e341 expref: 251 pid: 14162 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.934898:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.934901:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.934904:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.934916:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.934934:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.934949:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+250:2406:x1871938519959552:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 120us (1426us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.934955:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100412 00000100:00100000:0.0:1785220581.934969:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100413 00000100:00100000:0.0:1785220581.934975:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+249:2406:x1871938519959808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.934983:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.934988:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008f9711bc/0xf62499ab9278d4ff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e222 expref: 249 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.934998:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb8380, transno 4295069117 00010000:00010000:0.0:1785220581.934998:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008f9711bc/0xf62499ab9278d4ff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e222 expref: 249 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935001:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6e300, transno 4295069120 00000004:00080000:2.0:1785220581.935003:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb9500, transno 4295069123 00000004:00080000:2.0:1785220581.935005:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbbbb80, transno 4295069124 00000004:00080000:2.0:1785220581.935008:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb8a80, transno 4295069125 00010000:00010000:0.0:1785220581.935010:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008f9711bc/0xf62499ab9278d4ff lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e222 expref: 249 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935011:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6d500, transno 4295069128 00000004:00080000:2.0:1785220581.935013:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6c000, transno 4295069129 00000004:00080000:2.0:1785220581.935016:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0dca80, transno 4295069130 00000004:00080000:2.0:1785220581.935018:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa67100, transno 4295069133 00000004:00080000:2.0:1785220581.935020:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0dfb80, transno 4295069134 00010000:00010000:0.0:1785220581.935020:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7269:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.935023:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c2ad80, transno 4295069139 00010000:00010000:0.0:1785220581.935023:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7269:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.935025:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b349880, transno 4295069140 00010000:00010000:0.0:1785220581.935027:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935028:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c2a680, transno 4295069142 00000004:00080000:2.0:1785220581.935030:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ec380, transno 4295069144 00000004:00080000:2.0:1785220581.935033:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a5500, transno 4295069145 00000004:00080000:2.0:1785220581.935035:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c29180, transno 4295069146 00000004:00080000:2.0:1785220581.935038:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa65f80, transno 4295069147 00000004:00080000:2.0:1785220581.935040:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a4700, transno 4295069148 00010000:00010000:0.0:1785220581.935040:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935043:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a6a00, transno 4295069154 00000004:00080000:2.0:1785220581.935045:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96419180, transno 4295069155 00000004:00080000:2.0:1785220581.935048:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a5f80, transno 4295069156 00000004:00080000:2.0:1785220581.935050:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794e680, transno 4295069157 00000004:00080000:2.0:1785220581.935052:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794f480, transno 4295069159 00000004:00080000:2.0:1785220581.935055:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794d180, transno 4295069160 00010000:00010000:0.0:1785220581.935056:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.935058:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d793100, transno 4295069162 00000004:00080000:2.0:1785220581.935061:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e85c00, transno 4295069164 00000004:00080000:2.0:1785220581.935063:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e84a80, transno 4295069166 00000004:00080000:2.0:1785220581.935066:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c09880, transno 4295069170 00000100:00100000:0.0:1785220581.935068:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+248:2406:x1871938519959808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (1536us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.935069:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc1880, transno 4295069172 00000004:00080000:2.0:1785220581.935071:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc2d80, transno 4295069173 00000004:00080000:2.0:1785220581.935073:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc3800, transno 4295069174 00000100:00100000:0.0:1785220581.935073:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100413 00000004:00080000:2.0:1785220581.935076:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc1500, transno 4295069176 00000004:00080000:2.0:1785220581.935079:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c0b800, transno 4295069177 00000004:00080000:2.0:1785220581.935081:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c08700, transno 4295069180 00000004:00080000:2.0:1785220581.935084:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e85500, transno 4295069181 00000004:00080000:2.0:1785220581.935114:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e85f80, transno 4295069183 00000004:00080000:2.0:1785220581.935116:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794ca80, transno 4295069186 00000004:00080000:2.0:1785220581.935119:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794f800, transno 4295069188 00000004:00080000:2.0:1785220581.935220:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e86680, transno 4295069189 00000004:00080000:2.0:1785220581.935222:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794fb80, transno 4295069190 00000004:00080000:2.0:1785220581.935225:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56c700, transno 4295069193 00000004:00080000:2.0:1785220581.935227:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56c380, transno 4295069194 00000004:00080000:2.0:1785220581.935230:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91602a00, transno 4295069195 00000004:00080000:2.0:1785220581.935233:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b9b800, transno 4295069196 00000004:00080000:2.0:1785220581.935235:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e71500, transno 4295069200 00010000:00010000:1.0:1785220581.935235:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729c:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.935238:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef2680, transno 4295069201 00010000:00010000:1.0:1785220581.935239:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729c:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.935240:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e70000, transno 4295069204 00000004:00080000:2.0:1785220581.935243:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91601f80, transno 4295069205 00010000:00010000:1.0:1785220581.935244:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935246:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef0700, transno 4295069207 00000004:00080000:2.0:1785220581.935248:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef1880, transno 4295069208 00000004:00080000:2.0:1785220581.935251:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef3480, transno 4295069210 00010000:00010000:1.0:1785220581.935252:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935254:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef0e00, transno 4295069211 00000004:00080000:2.0:1785220581.935256:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef2d80, transno 4295069212 00000004:00080000:2.0:1785220581.935259:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef0000, transno 4295069213 00000004:00080000:2.0:1785220581.935262:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef1f80, transno 4295069216 00000004:00080000:2.0:1785220581.935264:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853f800, transno 4295069219 00000004:00080000:2.0:1785220581.935267:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e70e00, transno 4295069221 00000004:00080000:2.0:1785220581.935269:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853dc00, transno 4295069223 00010000:00010000:1.0:1785220581.935270:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.935272:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853e680, transno 4295069226 00000004:00080000:2.0:1785220581.935274:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853ea00, transno 4295069227 00000004:00080000:2.0:1785220581.935277:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef9f80, transno 4295069230 00000004:00080000:2.0:1785220581.935279:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99934000, transno 4295069232 00000004:00080000:2.0:1785220581.935281:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853c700, transno 4295069233 00000004:00080000:2.0:1785220581.935284:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853ce00, transno 4295069235 00000100:00100000:1.0:1785220581.935284:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+209:2407:x1871938519959296:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1242us (2010us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.935286:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853c380, transno 4295069237 00000004:00080000:2.0:1785220581.935289:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef9c00, transno 4295069238 00000004:00080000:2.0:1785220581.935291:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99935180, transno 4295069241 00000004:00080000:2.0:1785220581.935294:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99027800, transno 4295069243 00000004:00080000:2.0:1785220581.935296:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91603480, transno 4295069245 00000004:00080000:2.0:1785220581.935299:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91603800, transno 4295069246 00000004:00080000:2.0:1785220581.935301:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91603100, transno 4295069247 00000004:00080000:2.0:1785220581.935303:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91602300, transno 4295069251 00000004:00080000:2.0:1785220581.935306:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91601180, transno 4295069252 00000004:00080000:2.0:1785220581.935308:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef0a80, transno 4295069255 00000004:00080000:2.0:1785220581.935310:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9641b100, transno 4295069257 00000100:00100000:0.0:1785220581.935311:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100414 00000004:00080000:2.0:1785220581.935313:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96419500, transno 4295069259 00000004:00080000:2.0:1785220581.935315:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96418a80, transno 4295069260 00000004:00080000:2.0:1785220581.935318:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9641ad80, transno 4295069263 00000100:00100000:0.0:1785220581.935319:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90701c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+247:2405:x1871938519960064:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000004:00080000:2.0:1785220581.935320:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96419c00, transno 4295069264 00000004:00080000:2.0:1785220581.935323:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9641a680, transno 4295069267 00000004:00080000:2.0:1785220581.935325:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b34a680, transno 4295069271 00010000:00010000:0.0:1785220581.935327:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:2.0:1785220581.935328:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91600000, transno 4295069272 00000004:00080000:2.0:1785220581.935330:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c29f80, transno 4295069275 00010000:00010000:0.0:1785220581.935332:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000065bfbd4c/0xf62499ab9278d919 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e3d4 expref: 247 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935333:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb78380, transno 4295069278 00000004:00080000:2.0:1785220581.935335:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d2d80, transno 4295069279 00000004:00080000:2.0:1785220581.935338:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91600700, transno 4295069280 00000004:00080000:2.0:1785220581.935341:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d1180, transno 4295069282 00010000:00010000:0.0:1785220581.935341:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000065bfbd4c/0xf62499ab9278d919 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e3d4 expref: 247 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935343:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d1f80, transno 4295069288 00000004:00080000:2.0:1785220581.935346:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0ff480, transno 4295069289 00000004:00080000:2.0:1785220581.935349:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef9880, transno 4295069290 00000004:00080000:2.0:1785220581.935351:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91600a80, transno 4295069291 00010000:00010000:0.0:1785220581.935353:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726c:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.935354:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d2680, transno 4295069292 00010000:00010000:0.0:1785220581.935356:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726c:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.935357:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914fb80, transno 4295069294 00000004:00080000:2.0:1785220581.935359:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b29f80, transno 4295069296 00010000:00010000:0.0:1785220581.935360:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935362:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2b480, transno 4295069297 00000004:00080000:2.0:1785220581.935365:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efbb80, transno 4295069302 00000004:00080000:2.0:1785220581.935368:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e3800, transno 4295069304 00010000:00010000:0.0:1785220581.935368:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935370:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dc000, transno 4295069306 00000004:00080000:2.0:1785220581.935373:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964ddf80, transno 4295069308 00000004:00080000:2.0:1785220581.935375:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef8a80, transno 4295069309 00000004:00080000:2.0:1785220581.935377:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e2d80, transno 4295069312 00000004:00080000:2.0:1785220581.935379:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e0e00, transno 4295069315 00000004:00080000:2.0:1785220581.935382:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e1880, transno 4295069317 00000004:00080000:2.0:1785220581.935384:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1f100, transno 4295069318 00010000:00010000:0.0:1785220581.935384:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.935387:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1ed80, transno 4295069319 00000004:00080000:2.0:1785220581.935389:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e1500, transno 4295069321 00000004:00080000:2.0:1785220581.935392:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e1c00, transno 4295069322 00000004:00080000:2.0:1785220581.935394:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e0a80, transno 4295069323 00000100:00100000:0.0:1785220581.935396:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90701c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+247:2405:x1871938519960064:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (1848us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.935397:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c570a80, transno 4295069326 00000004:00080000:2.0:1785220581.935400:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c572300, transno 4295069327 00000004:00080000:2.0:1785220581.935402:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c570000, transno 4295069331 00000004:00080000:2.0:1785220581.935404:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c570380, transno 4295069332 00000004:00080000:2.0:1785220581.935406:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7ed80, transno 4295069334 00000004:00080000:2.0:1785220581.935413:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c571c00, transno 4295069337 00000004:00080000:2.0:1785220581.935416:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c570700, transno 4295069339 00000004:00080000:2.0:1785220581.935419:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906aaa00, transno 4295069340 00000004:00080000:2.0:1785220581.935421:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c572a00, transno 4295069341 00000004:00080000:2.0:1785220581.935424:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c573b80, transno 4295069345 00000100:00100000:1.0:1785220581.935425:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100406 00000004:00080000:2.0:1785220581.935426:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914e680, transno 4295069347 00000004:00080000:2.0:1785220581.935428:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c571180, transno 4295069348 00000004:00080000:2.0:1785220581.935430:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914c000, transno 4295069350 00000004:00080000:2.0:1785220581.935432:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914ca80, transno 4295069352 00000004:00080000:2.0:1785220581.935435:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d2d80, transno 4295069354 00000004:00080000:2.0:1785220581.935437:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bf100, transno 4295069357 00000004:00080000:2.0:1785220581.935439:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dce00, transno 4295069359 00000004:00080000:2.0:1785220581.935442:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93573100, transno 4295069361 00000004:00080000:2.0:1785220581.935444:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efb480, transno 4295069362 00000004:00080000:2.0:1785220581.935447:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bca80, transno 4295069363 00000004:00080000:2.0:1785220581.935449:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faaaa00, transno 4295069365 00000004:00080000:2.0:1785220581.935451:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa9f80, transno 4295069369 00000004:00080000:2.0:1785220581.935454:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa9180, transno 4295069370 00000004:00080000:2.0:1785220581.935457:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0cb480, transno 4295069372 00000004:00080000:2.0:1785220581.935459:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e0700, transno 4295069376 00000004:00080000:2.0:1785220581.935462:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc35180, transno 4295069378 00000004:00080000:2.0:1785220581.935464:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef6300, transno 4295069379 00000004:00080000:2.0:1785220581.935467:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e2680, transno 4295069380 00000004:00080000:2.0:1785220581.935469:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e1880, transno 4295069381 00000004:00080000:2.0:1785220581.935471:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96769500, transno 4295069385 00000004:00080000:2.0:1785220581.935474:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96768e00, transno 4295069386 00010000:00010000:1.0:1785220581.935475:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000004:00080000:2.0:1785220581.935477:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfad80, transno 4295069389 00000004:00080000:2.0:1785220581.935479:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb86300, transno 4295069393 00000004:00080000:2.0:1785220581.935481:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e2a00, transno 4295069394 00010000:00010000:1.0:1785220581.935481:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935484:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb85f80, transno 4295069396 00000004:00080000:2.0:1785220581.935486:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e1f80, transno 4295069397 00000004:00080000:2.0:1785220581.935489:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be1500, transno 4295069398 00010000:00010000:1.0:1785220581.935489:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.935491:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef4000, transno 4295069401 00010000:00010000:1.0:1785220581.935493:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935494:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef6a00, transno 4295069404 00000004:00080000:2.0:1785220581.935496:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95538700, transno 4295069405 00000004:00080000:2.0:1785220581.935499:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16bb80, transno 4295069407 00000004:00080000:2.0:1785220581.935501:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c169500, transno 4295069410 00010000:00010000:1.0:1785220581.935503:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935504:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c169180, transno 4295069411 00000004:00080000:2.0:1785220581.935506:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb3800, transno 4295069412 00000004:00080000:2.0:1785220581.935509:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb2a00, transno 4295069414 00000004:00080000:2.0:1785220581.935511:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c970700, transno 4295069416 00000004:00080000:2.0:1785220581.935513:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c971880, transno 4295069417 00000004:00080000:2.0:1785220581.935516:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b14380, transno 4295069422 00000004:00080000:2.0:1785220581.935518:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c971f80, transno 4295069423 00010000:00010000:1.0:1785220581.935520:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935521:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c972a00, transno 4295069425 00000004:00080000:2.0:1785220581.935524:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73b800, transno 4295069428 00000004:00080000:2.0:1785220581.935526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b4a80, transno 4295069430 00010000:00010000:1.0:1785220581.935528:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5167 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7268:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935529:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af8e00, transno 4295069432 00000004:00080000:2.0:1785220581.935531:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af9f80, transno 4295069435 00000004:00080000:2.0:1785220581.935534:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afa680, transno 4295069437 00000004:00080000:2.0:1785220581.935536:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa4700, transno 4295069438 00000004:00080000:2.0:1785220581.935539:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa6a00, transno 4295069440 00000004:00080000:2.0:1785220581.935541:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e0380, transno 4295069443 00000004:00080000:2.0:1785220581.935544:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e2300, transno 4295069444 00010000:00010000:1.0:1785220581.935545:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000004:00080000:2.0:1785220581.935546:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bec000, transno 4295069446 00010000:00010000:1.0:1785220581.935548:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935549:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97befb80, transno 4295069450 00000004:00080000:2.0:1785220581.935552:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97beca80, transno 4295069451 00010000:00010000:1.0:1785220581.935554:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.935555:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bedf80, transno 4295069453 00000004:00080000:2.0:1785220581.935557:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa6d80, transno 4295069455 00010000:00010000:1.0:1785220581.935557:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935559:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bed880, transno 4295069457 00000004:00080000:2.0:1785220581.935562:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93740000, transno 4295069458 00000004:00080000:2.0:1785220581.935564:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93740380, transno 4295069459 00010000:00010000:1.0:1785220581.935564:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935567:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91507100, transno 4295069461 00000004:00080000:2.0:1785220581.935569:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362b100, transno 4295069465 00000004:00080000:2.0:1785220581.935572:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93629880, transno 4295069467 00010000:00010000:1.0:1785220581.935572:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935574:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364b100, transno 4295069469 00000004:00080000:2.0:1785220581.935577:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d85ee3800, transno 4295069471 00010000:00010000:1.0:1785220581.935579:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5175 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.935580:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95450000, transno 4295069473 00000004:00080000:2.0:1785220581.935583:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d85ee0000, transno 4295069474 00000004:00080000:2.0:1785220581.935585:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362b800, transno 4295069475 00000004:00080000:2.0:1785220581.935587:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93740700, transno 4295069482 00000004:00080000:2.0:1785220581.935590:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1d880, transno 4295069483 00010000:00010000:1.0:1785220581.935590:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.935592:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.935593:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfca80, transno 4295069484 00010000:00010000:1.0:1785220581.935594:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.935595:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95451500, transno 4295069486 00000004:00080000:2.0:1785220581.935598:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93632680, transno 4295069488 00000004:00080000:2.0:1785220581.935601:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93631180, transno 4295069490 00000004:00080000:2.0:1785220581.935603:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbce00, transno 4295069491 00000004:00080000:2.0:1785220581.935606:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93630700, transno 4295069494 00000004:00080000:2.0:1785220581.935608:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbd500, transno 4295069497 00000004:00080000:2.0:1785220581.935610:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984e680, transno 4295069499 00000004:00080000:2.0:1785220581.935612:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984f800, transno 4295069500 00000004:00080000:2.0:1785220581.935615:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e0700, transno 4295069501 00000004:00080000:2.0:1785220581.935617:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e3800, transno 4295069503 00000004:00080000:2.0:1785220581.935619:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc1180, transno 4295069506 00000004:00080000:2.0:1785220581.935622:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e1880, transno 4295069507 00000004:00080000:2.0:1785220581.935624:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c970000, transno 4295069508 00000004:00080000:2.0:1785220581.935626:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e2d80, transno 4295069509 00000004:00080000:2.0:1785220581.935628:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e0a80, transno 4295069511 00000004:00080000:1.0:1785220581.935628:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a84000/2, rc 0, transno 4295070056 00000004:00080000:2.0:1785220581.935630:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c970e00, transno 4295069513 00000004:00080000:2.0:1785220581.935632:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c972300, transno 4295069518 00000100:00100000:1.0:1785220581.935634:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510700800:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220581.935635:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b15c00, transno 4295069522 00000004:00080000:2.0:1785220581.935637:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dff100, transno 4295069524 00000004:00080000:2.0:1785220581.935639:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc3800, transno 4295069526 00000004:00080000:2.0:1785220581.935642:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbd180, transno 4295069528 00000004:00080000:2.0:1785220581.935644:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b14e00, transno 4295069530 00000004:00080000:2.0:1785220581.935647:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfc700, transno 4295069532 00000004:00080000:2.0:1785220581.935649:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc3b80, transno 4295069533 00000004:00080000:2.0:1785220581.935652:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad8380, transno 4295069535 00000004:00080000:2.0:1785220581.935654:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfce00, transno 4295069536 00000004:00080000:2.0:1785220581.935657:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbe680, transno 4295069541 00000004:00080000:2.0:1785220581.935659:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbc000, transno 4295069542 00000004:00080000:2.0:1785220581.935662:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef7480, transno 4295069545 00000004:00080000:2.0:1785220581.935664:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e31c00, transno 4295069547 00000004:00080000:2.0:1785220581.935667:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef4700, transno 4295069548 00000004:00080000:2.0:1785220581.935669:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef5500, transno 4295069550 00000004:00080000:2.0:1785220581.935671:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e33100, transno 4295069551 00000004:00080000:2.0:1785220581.935674:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e33480, transno 4295069552 00000004:00080000:2.0:1785220581.935676:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986ea00, transno 4295069554 00000004:00080000:2.0:1785220581.935678:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98486300, transno 4295069559 00000004:00080000:2.0:1785220581.935681:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986f800, transno 4295069560 00000004:00080000:2.0:1785220581.935684:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e1c00, transno 4295069561 00000004:00080000:2.0:1785220581.935686:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986d180, transno 4295069563 00000004:00080000:2.0:1785220581.935689:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b45c00, transno 4295069564 00000004:00080000:2.0:1785220581.935692:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc0e00, transno 4295069568 00000004:00080000:2.0:1785220581.935695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e32d80, transno 4295069569 00000004:00080000:2.0:1785220581.935697:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad8700, transno 4295069571 00000004:00080000:2.0:1785220581.935700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfe300, transno 4295069576 00000004:00080000:2.0:1785220581.935703:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ada680, transno 4295069577 00000004:00080000:2.0:1785220581.935706:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e31180, transno 4295069578 00000004:00080000:2.0:1785220581.935709:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97957b80, transno 4295069579 00000004:00080000:2.0:1785220581.935711:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b16a00, transno 4295069581 00000004:00080000:2.0:1785220581.935714:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93629c00, transno 4295069583 00000004:00080000:2.0:1785220581.935717:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b47480, transno 4295069585 00000004:00080000:2.0:1785220581.935720:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93628a80, transno 4295069586 00000004:00080000:2.0:1785220581.935723:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bed80, transno 4295069591 00000004:00080000:2.0:1785220581.935725:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97956680, transno 4295069592 00000004:00080000:2.0:1785220581.935727:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97955500, transno 4295069596 00000004:00080000:2.0:1785220581.935730:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6dc00, transno 4295069598 00000004:00080000:2.0:1785220581.935732:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00f800, transno 4295069599 00000004:00080000:2.0:1785220581.935735:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bd880, transno 4295069600 00000004:00080000:2.0:1785220581.935738:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996e680, transno 4295069601 00000004:00080000:2.0:1785220581.935741:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996dc00, transno 4295069602 00000004:00080000:2.0:1785220581.935744:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0ab480, transno 4295069609 00000004:00080000:2.0:1785220581.935746:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49ea00, transno 4295069611 00000004:00080000:2.0:1785220581.935749:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49e680, transno 4295069612 00000004:00080000:2.0:1785220581.935751:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbce00, transno 4295069613 00000004:00080000:2.0:1785220581.935765:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735ce00, transno 4295069614 00000004:00080000:2.0:1785220581.935768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8df80, transno 4295069616 00000100:00100000:0.0:1785220581.935784:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100414 00010000:00010000:0.0:1785220581.935834:0:14150:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b087cbec/0xf62499ab9278f829 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f259 expref: 208 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.935846:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510702848:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.935866:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.935876:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.935883:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.935892:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.935900:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.935916:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.936046:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+21:3653:x1871938510701696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10720us (14179us total) trans 4295070643 rc 0/0 00000100:00100000:0.0:1785220581.936052:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284476 00010000:00010000:0.0:1785220581.936217:0:14262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015ca6e7b/0xf62499ab9278f93a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f275 expref: 207 pid: 14177 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.936230:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510703104:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.936243:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936250:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936256:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936265:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936271:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936285:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220581.936360:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+21:3655:x1871938510701952:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10847us (12541us total) trans 4295070644 rc 0/0 00000100:00100000:0.0:1785220581.936367:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284478 00010000:00010000:0.0:1785220581.936398:0:14172:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000588f8e63/0xf62499ab9278d62c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e2a7 expref: 246 pid: 14222 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.936410:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510703360:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.936423:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936431:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936438:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936444:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936450:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936469:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220581.936553:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d985fe450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1205:3653:x1871938510702208:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10828us (14133us total) trans 4295070057 rc 0/0 00000100:00100000:0.0:1785220581.936560:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284480 00010000:00010000:0.0:1785220581.936582:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936595:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936603:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936611:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936621:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220581.936692:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+21:3655:x1871938510700672:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15317us (15429us total) trans 4295070645 rc 0/0 00000100:00100000:0.0:1785220581.936699:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284472 00010000:00010000:0.0:1785220581.936721:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936730:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936737:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936744:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936770:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:0.0:1785220581.936836:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1205:3655:x1871938510702080:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11203us (12763us total) trans 4295070058 rc 0/0 00000100:00100000:0.0:1785220581.936842:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284479 00010000:00010000:0.0:1785220581.936863:0:14261:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936872:0:14261:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936881:0:14261:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936888:0:14261:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.936896:0:14261:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:0.0:1785220581.936967:0:14261:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1205:3653:x1871938510702336:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11589us (14599us total) trans 4295070059 rc 0/0 00000100:00100000:0.0:1785220581.936973:0:14261:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284477 00010000:00010000:0.0:1785220581.936995:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937003:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937016:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937024:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937032:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/7 locks to regular blp list, 7 blwis in pool 00000100:00100000:0.0:1785220581.937128:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1205:3653:x1871938510700160:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20989us (22688us total) trans 4295070060 rc 0/0 00000100:00100000:0.0:1785220581.937135:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284469 00010000:00010000:0.0:1785220581.937158:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937168:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937176:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937183:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937192:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/8 locks to regular blp list, 8 blwis in pool 00000100:00100000:0.0:1785220581.937256:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1205:3655:x1871938510700416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20531us (20582us total) trans 4295070061 rc 0/0 00000100:00100000:0.0:1785220581.937266:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284470 00010000:00010000:0.0:1785220581.937291:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937299:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937307:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937314:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937317:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8f800, transno 4295069617 00000004:00080000:2.0:1785220581.937323:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8d880, transno 4295069618 00010000:00010000:0.0:1785220581.937323:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/9 locks to regular blp list, 9 blwis in pool 00000004:00080000:2.0:1785220581.937326:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92165f80, transno 4295069619 00000004:00080000:2.0:1785220581.937329:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f0e00, transno 4295069622 00000004:00080000:2.0:1785220581.937331:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f1500, transno 4295069623 00000004:00080000:2.0:1785220581.937334:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92164700, transno 4295069625 00000004:00080000:2.0:1785220581.937336:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dc000, transno 4295069628 00000004:00080000:2.0:1785220581.937339:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f1880, transno 4295069630 00000004:00080000:2.0:1785220581.937342:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977df480, transno 4295069632 00000004:00080000:2.0:1785220581.937344:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a5880, transno 4295069634 00000004:00080000:2.0:1785220581.937347:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e87b80, transno 4295069636 00000004:00080000:2.0:1785220581.937350:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f57800, transno 4295069637 00000004:00080000:2.0:1785220581.937352:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a6300, transno 4295069640 00000004:00080000:2.0:1785220581.937355:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735d180, transno 4295069641 00000004:00080000:2.0:1785220581.937357:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f56a00, transno 4295069643 00000004:00080000:2.0:1785220581.937360:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe9180, transno 4295069645 00000004:00080000:2.0:1785220581.937362:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a5c00, transno 4295069646 00000004:00080000:2.0:1785220581.937365:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b44380, transno 4295069650 00000004:00080000:2.0:1785220581.937368:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1ea00, transno 4295069651 00000004:00080000:2.0:1785220581.937370:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97beed80, transno 4295069654 00000004:00080000:2.0:1785220581.937373:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991be680, transno 4295069657 00000004:00080000:2.0:1785220581.937376:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bf100, transno 4295069660 00000004:00080000:2.0:1785220581.937378:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00c000, transno 4295069662 00000004:00080000:2.0:1785220581.937381:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bf480, transno 4295069663 00000004:00080000:2.0:1785220581.937383:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c970380, transno 4295069664 00000004:00080000:2.0:1785220581.937386:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bed80, transno 4295069665 00000004:00080000:2.0:1785220581.937388:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca9f80, transno 4295069669 00000004:00080000:2.0:1785220581.937391:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bc000, transno 4295069671 00000100:00100000:0.0:1785220581.937391:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+21:3655:x1871938510701056:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14579us (15596us total) trans 4295070646 rc 0/0 00000004:00080000:2.0:1785220581.937394:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b9a300, transno 4295069673 00000004:00080000:2.0:1785220581.937397:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e73100, transno 4295069675 00000100:00100000:0.0:1785220581.937397:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284474 00000004:00080000:2.0:1785220581.937399:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98412680, transno 4295069676 00000004:00080000:2.0:1785220581.937402:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0df800, transno 4295069677 00000004:00080000:2.0:1785220581.937405:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5f480, transno 4295069679 00000004:00080000:2.0:1785220581.937407:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5f800, transno 4295069682 00000004:00080000:2.0:1785220581.937410:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5e300, transno 4295069683 00000004:00080000:2.0:1785220581.937412:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5ce00, transno 4295069686 00000004:00080000:2.0:1785220581.937415:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856e300, transno 4295069688 00000004:00080000:2.0:1785220581.937417:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6f800, transno 4295069690 00010000:00010000:0.0:1785220581.937418:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 8 total in blp. (8 blwis in pool) 00000004:00080000:2.0:1785220581.937420:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6f480, transno 4295069691 00000004:00080000:2.0:1785220581.937422:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87d46300, transno 4295069698 00010000:00010000:0.0:1785220581.937423:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937425:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99375880, transno 4295069700 00000004:00080000:2.0:1785220581.937428:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8dc00, transno 4295069701 00000004:00080000:2.0:1785220581.937431:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8c380, transno 4295069703 00010000:00010000:0.0:1785220581.937431:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937433:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6ea00, transno 4295069704 00010000:00010000:0.0:1785220581.937435:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937436:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8f100, transno 4295069705 00000004:00080000:2.0:1785220581.937439:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be3100, transno 4295069706 00000004:00080000:2.0:1785220581.937441:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be3800, transno 4295069707 00000004:00080000:2.0:1785220581.937444:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98626a00, transno 4295069714 00010000:00010000:0.0:1785220581.937444:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937446:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93741f80, transno 4295069715 00000004:00080000:2.0:1785220581.937449:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98625c00, transno 4295069717 00000004:00080000:2.0:1785220581.937452:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93741c00, transno 4295069718 00000004:00080000:2.0:1785220581.937454:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95452a00, transno 4295069719 00000004:00080000:2.0:1785220581.937457:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d85ee3480, transno 4295069720 00000004:00080000:2.0:1785220581.937459:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af8380, transno 4295069721 00010000:00010000:0.0:1785220581.937461:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937462:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af9180, transno 4295069726 00000004:00080000:2.0:1785220581.937464:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ceb800, transno 4295069730 00000004:00080000:2.0:1785220581.937467:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce9880, transno 4295069731 00010000:00010000:0.0:1785220581.937468:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a518a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937470:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce8700, transno 4295069732 00000004:00080000:2.0:1785220581.937472:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ceb480, transno 4295069733 00000004:00080000:2.0:1785220581.937474:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ceb100, transno 4295069735 00000004:00080000:2.0:1785220581.937476:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c18000, transno 4295069738 00000004:00080000:2.0:1785220581.937478:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9adb1880, transno 4295069741 00000004:00080000:2.0:1785220581.937481:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984d500, transno 4295069742 00000004:00080000:2.0:1785220581.937484:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984e300, transno 4295069743 00010000:00010000:0.0:1785220581.937485:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 7 total in blp. (7 blwis in pool) 00000004:00080000:2.0:1785220581.937487:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676b100, transno 4295069747 00000004:00080000:2.0:1785220581.937489:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984d880, transno 4295069751 00010000:00010000:0.0:1785220581.937489:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937492:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95421880, transno 4295069752 00000004:00080000:2.0:1785220581.937494:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95423480, transno 4295069754 00010000:00010000:0.0:1785220581.937494:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ada00 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937497:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95421500, transno 4295069755 00010000:00010000:0.0:1785220581.937498:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937501:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93630a80, transno 4295069757 00000004:00080000:2.0:1785220581.937504:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95420000, transno 4295069759 00010000:00010000:0.0:1785220581.937505:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937506:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bee680, transno 4295069761 00000004:00080000:2.0:1785220581.937508:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d639180, transno 4295069763 00000004:00080000:2.0:1785220581.937511:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d638e00, transno 4295069764 00000004:00080000:2.0:1785220581.937513:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d639880, transno 4295069765 00010000:00010000:0.0:1785220581.937513:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937516:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93571500, transno 4295069766 00000004:00080000:2.0:1785220581.937518:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d639500, transno 4295069770 00000004:00080000:2.0:1785220581.937520:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00ed80, transno 4295069773 00010000:00010000:0.0:1785220581.937520:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5198 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937523:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb87b80, transno 4295069775 00000004:00080000:2.0:1785220581.937526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95448a80, transno 4295069776 00000004:00080000:2.0:1785220581.937529:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c0a300, transno 4295069777 00010000:00010000:0.0:1785220581.937530:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 6 total in blp. (6 blwis in pool) 00000004:00080000:2.0:1785220581.937532:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d638a80, transno 4295069778 00000004:00080000:2.0:1785220581.937534:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91173800, transno 4295069779 00010000:00010000:0.0:1785220581.937534:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937537:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c62300, transno 4295069781 00000004:00080000:2.0:1785220581.937540:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91170a80, transno 4295069782 00010000:00010000:0.0:1785220581.937540:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937543:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92166300, transno 4295069787 00010000:00010000:0.0:1785220581.937543:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937546:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90407800, transno 4295069791 00000004:00080000:2.0:1785220581.937549:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ac700, transno 4295069793 00010000:00010000:0.0:1785220581.937550:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937551:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90406a00, transno 4295069795 00000004:00080000:2.0:1785220581.937554:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90405500, transno 4295069796 00000004:00080000:2.0:1785220581.937556:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90406d80, transno 4295069797 00010000:00010000:0.0:1785220581.937558:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937559:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95539f80, transno 4295069798 00000004:00080000:2.0:1785220581.937561:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90406680, transno 4295069799 00000004:00080000:2.0:1785220581.937563:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99024380, transno 4295069803 00010000:00010000:0.0:1785220581.937564:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a51a6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937566:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbba680, transno 4295069806 00000004:00080000:2.0:1785220581.937568:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480e680, transno 4295069811 00000004:00080000:2.0:1785220581.937571:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73aa00, transno 4295069812 00000004:00080000:2.0:1785220581.937574:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480ce00, transno 4295069813 00010000:00010000:0.0:1785220581.937574:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00000004:00080000:2.0:1785220581.937576:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d3480, transno 4295069814 00010000:00010000:0.0:1785220581.937577:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937579:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480c000, transno 4295069816 00000004:00080000:2.0:1785220581.937581:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfeb800, transno 4295069818 00000004:00080000:2.0:1785220581.937584:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87d45880, transno 4295069819 00010000:00010000:0.0:1785220581.937584:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937586:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adb480, transno 4295069825 00010000:00010000:0.0:1785220581.937587:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937589:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856df80, transno 4295069826 00000004:00080000:2.0:1785220581.937591:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad9880, transno 4295069828 00010000:00010000:0.0:1785220581.937593:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937594:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a7800, transno 4295069829 00000004:00080000:2.0:1785220581.937596:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ed180, transno 4295069830 00000004:00080000:2.0:1785220581.937599:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a6680, transno 4295069831 00000004:00080000:2.0:1785220581.937601:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc2680, transno 4295069833 00010000:00010000:0.0:1785220581.937601:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937603:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c573100, transno 4295069837 00000004:00080000:2.0:1785220581.937606:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977ded80, transno 4295069838 00000004:00080000:2.0:1785220581.937609:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad0000, transno 4295069841 00010000:00010000:0.0:1785220581.937609:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a516e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937611:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad3800, transno 4295069842 00000004:00080000:2.0:1785220581.937613:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad3100, transno 4295069843 00000004:00080000:2.0:1785220581.937616:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad1880, transno 4295069844 00000004:00080000:2.0:1785220581.937619:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb86680, transno 4295069845 00010000:00010000:0.0:1785220581.937620:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00000004:00080000:2.0:1785220581.937622:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914f100, transno 4295069848 00010000:00010000:0.0:1785220581.937622:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937624:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97589c00, transno 4295069850 00000004:00080000:2.0:1785220581.937627:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcaad80, transno 4295069851 00010000:00010000:0.0:1785220581.937628:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937629:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bf480, transno 4295069857 00010000:00010000:0.0:1785220581.937631:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937632:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bf800, transno 4295069858 00000004:00080000:2.0:1785220581.937634:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bea00, transno 4295069859 00000004:00080000:2.0:1785220581.937636:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe8700, transno 4295069860 00010000:00010000:0.0:1785220581.937638:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937639:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfead80, transno 4295069861 00000004:00080000:2.0:1785220581.937641:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de6a00, transno 4295069863 00000004:00080000:2.0:1785220581.937644:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e772a00, transno 4295069868 00010000:00010000:0.0:1785220581.937646:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937647:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040c380, transno 4295069870 00000004:00080000:2.0:1785220581.937649:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e771180, transno 4295069873 00000004:00080000:2.0:1785220581.937652:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c0aa00, transno 4295069874 00010000:00010000:0.0:1785220581.937652:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a519f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7269:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937654:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040ca80, transno 4295069875 00000004:00080000:2.0:1785220581.937656:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73b480, transno 4295069876 00000004:00080000:2.0:1785220581.937659:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af5880, transno 4295069878 00000004:00080000:2.0:1785220581.937661:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95539500, transno 4295069882 00000004:00080000:2.0:1785220581.937663:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d0e00, transno 4295069884 00010000:00010000:0.0:1785220581.937664:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00000004:00080000:2.0:1785220581.937666:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553b100, transno 4295069885 00010000:00010000:0.0:1785220581.937667:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937669:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c168700, transno 4295069891 00000004:00080000:2.0:1785220581.937672:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86310700, transno 4295069893 00010000:00010000:0.0:1785220581.937673:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937674:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95473b80, transno 4295069896 00010000:00010000:0.0:1785220581.937676:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937677:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95471c00, transno 4295069897 00000004:00080000:2.0:1785220581.937680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98581500, transno 4295069898 00000004:00080000:2.0:1785220581.937682:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95471180, transno 4295069900 00010000:00010000:0.0:1785220581.937683:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937684:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95472d80, transno 4295069901 00000004:00080000:2.0:1785220581.937687:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf9500, transno 4295069903 00000004:00080000:2.0:1785220581.937689:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95471880, transno 4295069906 00010000:00010000:0.0:1785220581.937691:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937692:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95472680, transno 4295069907 00000004:00080000:2.0:1785220581.937695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95472a00, transno 4295069913 00000004:00080000:2.0:1785220581.937698:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979bb100, transno 4295069915 00010000:00010000:0.0:1785220581.937698:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a5191 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86312d80, transno 4295069916 00000004:00080000:2.0:1785220581.937703:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95471500, transno 4295069917 00000004:00080000:2.0:1785220581.937705:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86312680, transno 4295069918 00000004:00080000:2.0:1785220581.937706:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86311880, transno 4295069919 00000004:00080000:2.0:1785220581.937709:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe9f80, transno 4295069920 00010000:00010000:0.0:1785220581.937709:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00000004:00080000:2.0:1785220581.937711:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb37b80, transno 4295069923 00010000:00010000:0.0:1785220581.937712:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937714:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e770e00, transno 4295069924 00000004:00080000:2.0:1785220581.937716:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e772d80, transno 4295069926 00010000:00010000:0.0:1785220581.937717:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937719:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d639f80, transno 4295069929 00010000:00010000:0.0:1785220581.937720:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937721:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae69f80, transno 4295069930 00000004:00080000:2.0:1785220581.937724:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf8380, transno 4295069932 00000004:00080000:2.0:1785220581.937726:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6b480, transno 4295069934 00010000:00010000:0.0:1785220581.937726:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937729:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d63b100, transno 4295069935 00000004:00080000:2.0:1785220581.937731:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6a300, transno 4295069937 00000004:00080000:2.0:1785220581.937733:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae69880, transno 4295069939 00010000:00010000:0.0:1785220581.937735:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937736:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae69c00, transno 4295069942 00000004:00080000:2.0:1785220581.937738:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d1c00, transno 4295069944 00000004:00080000:2.0:1785220581.937741:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93648380, transno 4295069945 00010000:00010000:0.0:1785220581.937742:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5159 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7266:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937743:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d1f80, transno 4295069946 00000004:00080000:2.0:1785220581.937745:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d1880, transno 4295069947 00000004:00080000:2.0:1785220581.937748:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae69500, transno 4295069948 00000004:00080000:2.0:1785220581.937750:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d1500, transno 4295069952 00000004:00080000:2.0:1785220581.937752:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d3100, transno 4295069954 00000004:00080000:2.0:1785220581.937768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93631880, transno 4295069955 00010000:00010000:0.0:1785220581.937768:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000004:00080000:2.0:1785220581.937771:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040e680, transno 4295069956 00010000:00010000:0.0:1785220581.937771:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937773:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040e300, transno 4295069961 00000004:00080000:2.0:1785220581.937776:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c82a00, transno 4295069962 00010000:00010000:0.0:1785220581.937777:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00000004:00080000:2.0:1785220581.937779:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfaa00, transno 4295069964 00010000:00010000:0.0:1785220581.937780:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937782:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa8a80, transno 4295069965 00000004:00080000:2.0:1785220581.937785:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040df80, transno 4295069967 00000004:00080000:2.0:1785220581.937787:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95472300, transno 4295069970 00010000:00010000:0.0:1785220581.937787:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937790:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95470000, transno 4295069971 00000004:00080000:2.0:1785220581.937793:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95473480, transno 4295069972 00000004:00080000:2.0:1785220581.937796:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97956d80, transno 4295069977 00010000:00010000:0.0:1785220581.937797:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937798:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa8380, transno 4295069978 00000004:00080000:2.0:1785220581.937801:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfa680, transno 4295069979 00000004:00080000:2.0:1785220581.937803:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e771880, transno 4295069980 00010000:00010000:0.0:1785220581.937803:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a5160 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7267:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937806:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859bb80, transno 4295069981 00000004:00080000:2.0:1785220581.937808:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98598e00, transno 4295069984 00000004:00080000:2.0:1785220581.937810:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859a300, transno 4295069985 00000004:00080000:2.0:1785220581.937813:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98582a00, transno 4295069986 00010000:00010000:0.0:1785220581.937813:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000004:00080000:2.0:1785220581.937815:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98580a80, transno 4295069987 00010000:00010000:0.0:1785220581.937816:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.937817:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98582300, transno 4295069988 00000004:00080000:2.0:1785220581.937819:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5a680, transno 4295069989 00010000:00010000:0.0:1785220581.937822:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.937825:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937838:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937847:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937854:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a517c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.937867:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.937869:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.937870:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.937878:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.937884:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.937920:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c3800/2, rc 0, transno 4295070057 00000100:00100000:0.0:1785220581.937926:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510702208:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.937935:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c1180/2, rc 0, transno 4295070059 00000100:00100000:0.0:1785220581.937939:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510702336:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220581.938247:0:14252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000065bfbd4c/0xf62499ab9278d919 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e3d4 expref: 245 pid: 14159 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.938259:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510703488:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.938278:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.938288:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.938295:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.938310:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.938318:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.938341:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.938437:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1205:3655:x1871938510702464:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12441us (12621us total) trans 4295070062 rc 0/0 00000100:00100000:3.0:1785220581.938443:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284481 00000004:00080000:2.0:1785220581.938604:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937aa00, transno 4295069990 00000004:00080000:2.0:1785220581.938607:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad2a00, transno 4295069991 00000004:00080000:2.0:1785220581.938610:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98580380, transno 4295069992 00000004:00080000:2.0:1785220581.938612:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93760000, transno 4295069993 00000004:00080000:2.0:1785220581.938615:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a3800, transno 4295069994 00000004:00080000:2.0:1785220581.938618:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5b100, transno 4295069995 00000004:00080000:2.0:1785220581.938620:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5a300, transno 4295069996 00000004:00080000:2.0:1785220581.938622:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92a680, transno 4295069997 00000004:00080000:2.0:1785220581.938624:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c928700, transno 4295069998 00000004:00080000:2.0:1785220581.938626:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c928380, transno 4295069999 00000004:00080000:2.0:1785220581.938629:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6ad80, transno 4295070000 00000004:00080000:2.0:1785220581.938631:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbea00, transno 4295070001 00000004:00080000:2.0:1785220581.938634:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98598380, transno 4295070002 00000004:00080000:2.0:1785220581.938637:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99378e00, transno 4295070003 00000004:00080000:2.0:1785220581.938640:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95473100, transno 4295070004 00000004:00080000:2.0:1785220581.938642:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937bb80, transno 4295070005 00000004:00080000:2.0:1785220581.938645:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99379180, transno 4295070006 00000004:00080000:2.0:1785220581.938647:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937b800, transno 4295070007 00000004:00080000:2.0:1785220581.938650:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98599180, transno 4295070008 00000004:00080000:2.0:1785220581.938652:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98599f80, transno 4295070009 00000004:00080000:2.0:1785220581.938655:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859b480, transno 4295070010 00000004:00080000:2.0:1785220581.938657:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859aa00, transno 4295070011 00000004:00080000:2.0:1785220581.938660:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98598000, transno 4295070012 00000004:00080000:2.0:1785220581.938662:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98580700, transno 4295070013 00000004:00080000:2.0:1785220581.938664:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81218700, transno 4295070014 00000004:00080000:2.0:1785220581.938666:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98582680, transno 4295070015 00000004:00080000:2.0:1785220581.938669:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98581880, transno 4295070016 00000004:00080000:2.0:1785220581.938671:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98582d80, transno 4295070017 00000004:00080000:2.0:1785220581.938673:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93760380, transno 4295070018 00000004:00080000:2.0:1785220581.938675:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93760700, transno 4295070019 00000004:00080000:2.0:1785220581.938677:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93760a80, transno 4295070020 00000004:00080000:2.0:1785220581.938680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761180, transno 4295070021 00000004:00080000:2.0:1785220581.938682:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93763b80, transno 4295070022 00000004:00080000:2.0:1785220581.938685:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761500, transno 4295070023 00000004:00080000:2.0:1785220581.938687:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93763800, transno 4295070024 00000004:00080000:2.0:1785220581.938690:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937ad80, transno 4295070025 00000004:00080000:2.0:1785220581.938692:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97588000, transno 4295070026 00000004:00080000:2.0:1785220581.938695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5b100, transno 4295070027 00000004:00080000:2.0:1785220581.938697:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99378a80, transno 4295070028 00000004:00080000:2.0:1785220581.938699:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99378380, transno 4295070029 00000004:00080000:2.0:1785220581.938702:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99378700, transno 4295070030 00000004:00080000:2.0:1785220581.938704:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbdf80, transno 4295070031 00000004:00080000:2.0:1785220581.938706:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a0000, transno 4295070032 00000004:00080000:2.0:1785220581.938709:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b0000, transno 4295070033 00000004:00080000:2.0:1785220581.938712:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa58e00, transno 4295070034 00000004:00080000:2.0:1785220581.938714:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95538e00, transno 4295070035 00000004:00080000:2.0:1785220581.938717:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b2680, transno 4295070036 00000004:00080000:2.0:1785220581.938719:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b0700, transno 4295070037 00000004:00080000:2.0:1785220581.938721:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b1500, transno 4295070038 00000004:00080000:2.0:1785220581.938723:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b2a00, transno 4295070039 00000004:00080000:2.0:1785220581.938726:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86312300, transno 4295070040 00000004:00080000:2.0:1785220581.938728:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86313480, transno 4295070041 00000004:00080000:2.0:1785220581.938731:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998ddc00, transno 4295070042 00000004:00080000:2.0:1785220581.938734:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121b100, transno 4295070043 00000004:00080000:2.0:1785220581.938736:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998ddf80, transno 4295070044 00000004:00080000:2.0:1785220581.938739:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121bb80, transno 4295070045 00000004:00080000:2.0:1785220581.938741:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998de680, transno 4295070046 00000004:00080000:2.0:1785220581.938743:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95470e00, transno 4295070047 00000004:00080000:2.0:1785220581.938746:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121aa00, transno 4295070048 00000004:00080000:2.0:1785220581.938748:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81219c00, transno 4295070049 00000004:00080000:2.0:1785220581.938750:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121b800, transno 4295070050 00000004:00080000:2.0:1785220581.938752:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad2680, transno 4295070051 00000004:00080000:2.0:1785220581.938764:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121b480, transno 4295070052 00000004:00080000:2.0:1785220581.938767:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81218e00, transno 4295070053 00000004:00080000:2.0:1785220581.938769:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a69880, transno 4295070054 00000004:00080000:2.0:1785220581.938772:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96667b80, transno 4295070055 00010000:00010000:1.0:1785220581.938890:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.938896:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.938905:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.938908:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.938916:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.938929:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.938935:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a51ad lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.938952:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.938953:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.938955:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.938971:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510704000:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.939007:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.939037:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510704000 00000100:00100000:1.0:1785220581.939044:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510704000 00000100:00100000:1.0:1785220581.939106:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284484 00000100:00100000:1.0:1785220581.939116:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1205:3652:x1871938510704000:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.939139:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29293 count 1 00010000:00010000:1.0:1785220581.939158:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939174:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e3d0afd1/0xf62499ab9278d990 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e47c expref: 244 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939182:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x726d:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.939186:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.939190:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939215:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e3d0afd1/0xf62499ab9278d990 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e47c expref: 244 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939227:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e3d0afd1/0xf62499ab9278d990 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e47c expref: 244 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.939239:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510704512:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.939288:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.939309:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220581.939315:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510704128:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.939340:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.939362:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510704128 00000100:00100000:1.0:1785220581.939366:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510704128 00000100:00100000:1.0:1785220581.939389:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284485 00000100:00100000:1.0:1785220581.939395:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1206:3652:x1871938510704128:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.939407:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29294 count 1 00010000:00010000:1.0:1785220581.939418:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939430:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003b69da5c/0xf62499ab9278da9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4f3 expref: 244 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939436:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x726e:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.939439:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.939442:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939460:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003b69da5c/0xf62499ab9278da9a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4f3 expref: 244 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.939470:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003b69da5c/0xf62499ab9278da9a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4f3 expref: 244 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.939479:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510704640:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.939503:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdb40 going to sleep for 11 seconds 00000004:00080000:2.0:1785220581.939513:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6bb80, transno 4295070056 00000004:00080000:2.0:1785220581.939519:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937b480, transno 4295070057 00000004:00080000:2.0:1785220581.939521:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6b100, transno 4295070058 00000004:00080000:2.0:1785220581.939524:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5b480, transno 4295070059 00000004:00080000:2.0:1785220581.939526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96664e00, transno 4295070060 00000004:00080000:2.0:1785220581.939528:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81218a80, transno 4295070061 00000004:00080000:2.0:1785220581.939530:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93762680, transno 4295070062 00000004:00080000:2.0:1785220581.939533:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a69c00, transno 4295070063 00000004:00080000:2.0:1785220581.939535:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93762a00, transno 4295070064 00000004:00080000:2.0:1785220581.939538:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98583800, transno 4295070065 00000004:00080000:2.0:1785220581.939540:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040b100, transno 4295070066 00000004:00080000:2.0:1785220581.939542:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040a680, transno 4295070067 00000004:00080000:2.0:1785220581.939544:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59fb80, transno 4295070068 00000004:00080000:2.0:1785220581.939546:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59dc00, transno 4295070069 00000004:00080000:2.0:1785220581.939549:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59c700, transno 4295070070 00000004:00080000:2.0:1785220581.939551:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90409500, transno 4295070071 00000004:00080000:2.0:1785220581.939554:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93760e00, transno 4295070072 00000004:00080000:2.0:1785220581.939556:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59d880, transno 4295070073 00000004:00080000:2.0:1785220581.939559:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c3800, transno 4295070074 00000004:00080000:2.0:1785220581.939561:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c1c00, transno 4295070075 00000004:00080000:2.0:1785220581.939564:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c0700, transno 4295070076 00000004:00080000:2.0:1785220581.939566:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c1180, transno 4295070077 00000004:00080000:2.0:1785220581.939568:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c1880, transno 4295070078 00000004:00080000:2.0:1785220581.939570:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c3480, transno 4295070079 00000004:00080000:2.0:1785220581.939572:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c3100, transno 4295070080 00000004:00080000:2.0:1785220581.939574:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c2680, transno 4295070081 00000004:00080000:2.0:1785220581.939577:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae90a80, transno 4295070082 00000004:00080000:2.0:1785220581.939579:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae92680, transno 4295070083 00000004:00080000:2.0:1785220581.939582:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99271500, transno 4295070084 00000004:00080000:2.0:1785220581.939585:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90409180, transno 4295070085 00000004:00080000:2.0:1785220581.939587:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040bb80, transno 4295070086 00000004:00080000:2.0:1785220581.939590:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99273800, transno 4295070087 00000004:00080000:2.0:1785220581.939592:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae93b80, transno 4295070088 00000004:00080000:2.0:1785220581.939594:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae92300, transno 4295070089 00000004:00080000:2.0:1785220581.939597:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99271f80, transno 4295070090 00000004:00080000:2.0:1785220581.939600:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99273100, transno 4295070091 00000004:00080000:2.0:1785220581.939602:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99272d80, transno 4295070092 00000004:00080000:2.0:1785220581.939605:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99272300, transno 4295070093 00000004:00080000:2.0:1785220581.939607:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c2a00, transno 4295070094 00000004:00080000:2.0:1785220581.939610:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99271c00, transno 4295070095 00000004:00080000:2.0:1785220581.939612:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99270e00, transno 4295070096 00000004:00080000:2.0:1785220581.939614:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c0000, transno 4295070097 00000004:00080000:2.0:1785220581.939617:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99270a80, transno 4295070098 00000004:00080000:2.0:1785220581.939620:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59f100, transno 4295070099 00000004:00080000:2.0:1785220581.939622:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59e300, transno 4295070100 00000004:00080000:2.0:1785220581.939625:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59d180, transno 4295070101 00000004:00080000:2.0:1785220581.939627:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae91180, transno 4295070102 00000004:00080000:2.0:1785220581.939630:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9363b100, transno 4295070103 00000004:00080000:2.0:1785220581.939632:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9363bb80, transno 4295070104 00000004:00080000:2.0:1785220581.939635:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9363ad80, transno 4295070105 00000004:00080000:2.0:1785220581.939638:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9363b800, transno 4295070106 00000004:00080000:2.0:1785220581.939640:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a262680, transno 4295070107 00000004:00080000:2.0:1785220581.939643:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae93480, transno 4295070108 00000004:00080000:2.0:1785220581.939645:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a263480, transno 4295070109 00000004:00080000:2.0:1785220581.939647:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93639180, transno 4295070110 00000004:00080000:2.0:1785220581.939650:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae90000, transno 4295070111 00000004:00080000:2.0:1785220581.939652:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99271180, transno 4295070112 00000004:00080000:2.0:1785220581.939655:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b58a80, transno 4295070113 00000004:00080000:2.0:1785220581.939658:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5a680, transno 4295070114 00000004:00080000:2.0:1785220581.939660:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a1880, transno 4295070115 00000004:00080000:2.0:1785220581.939662:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98583480, transno 4295070116 00000004:00080000:2.0:1785220581.939665:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a0380, transno 4295070117 00000004:00080000:2.0:1785220581.939667:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98581f80, transno 4295070118 00000004:00080000:2.0:1785220581.939670:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59c380, transno 4295070119 00000004:00080000:2.0:1785220581.939672:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f59f80, transno 4295070120 00000004:00080000:2.0:1785220581.939674:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b3480, transno 4295070121 00000004:00080000:2.0:1785220581.939676:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b1880, transno 4295070122 00000004:00080000:2.0:1785220581.939680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90408000, transno 4295070123 00000004:00080000:2.0:1785220581.939682:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90408380, transno 4295070124 00000004:00080000:2.0:1785220581.939684:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040aa00, transno 4295070125 00000004:00080000:2.0:1785220581.939686:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040ad80, transno 4295070126 00000004:00080000:2.0:1785220581.939688:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad2d80, transno 4295070127 00000004:00080000:2.0:1785220581.939690:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad3480, transno 4295070128 00000004:00080000:2.0:1785220581.939692:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad2300, transno 4295070129 00000004:00080000:2.0:1785220581.939695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c680e00, transno 4295070130 00000004:00080000:2.0:1785220581.939698:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c681500, transno 4295070131 00000004:00080000:2.0:1785220581.939700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c683800, transno 4295070132 00000004:00080000:2.0:1785220581.939703:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c680a80, transno 4295070133 00000004:00080000:2.0:1785220581.939705:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c682300, transno 4295070134 00000004:00080000:2.0:1785220581.939707:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c683100, transno 4295070135 00000004:00080000:2.0:1785220581.939710:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c680000, transno 4295070136 00000004:00080000:2.0:1785220581.939713:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad1180, transno 4295070137 00000004:00080000:2.0:1785220581.939715:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c682a00, transno 4295070138 00000004:00080000:2.0:1785220581.939717:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998df480, transno 4295070139 00000004:00080000:2.0:1785220581.939719:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b59880, transno 4295070140 00000004:00080000:2.0:1785220581.939722:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81219f80, transno 4295070141 00000004:00080000:2.0:1785220581.939724:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5ad80, transno 4295070142 00000004:00080000:2.0:1785220581.939726:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f59500, transno 4295070143 00000004:00080000:2.0:1785220581.939730:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f8e00, transno 4295070144 00000004:00080000:2.0:1785220581.939732:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae91880, transno 4295070145 00000004:00080000:2.0:1785220581.939734:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5ad80, transno 4295070146 00000004:00080000:2.0:1785220581.939737:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f9500, transno 4295070147 00000004:00080000:2.0:1785220581.939739:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f9180, transno 4295070148 00000004:00080000:2.0:1785220581.939741:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59ea00, transno 4295070149 00000004:00080000:2.0:1785220581.939743:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fb480, transno 4295070150 00000004:00080000:2.0:1785220581.939745:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93639f80, transno 4295070151 00000004:00080000:2.0:1785220581.939747:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9363a680, transno 4295070152 00000004:00080000:2.0:1785220581.939751:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99273b80, transno 4295070153 00000004:00080000:2.0:1785220581.939753:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fbb80, transno 4295070154 00000004:00080000:2.0:1785220581.939765:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93638a80, transno 4295070155 00000004:00080000:2.0:1785220581.939768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a261c00, transno 4295070156 00000004:00080000:2.0:1785220581.939771:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d311180, transno 4295070157 00000004:00080000:2.0:1785220581.939773:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fca80, transno 4295070158 00000004:00080000:2.0:1785220581.939776:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99379880, transno 4295070159 00000004:00080000:2.0:1785220581.939778:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a261880, transno 4295070160 00000004:00080000:2.0:1785220581.939780:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fea00, transno 4295070161 00000004:00080000:2.0:1785220581.939783:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d312a00, transno 4295070162 00000004:00080000:2.0:1785220581.939785:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d313480, transno 4295070163 00000004:00080000:2.0:1785220581.939788:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a260a80, transno 4295070164 00000004:00080000:2.0:1785220581.939790:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a260700, transno 4295070165 00000004:00080000:2.0:1785220581.939792:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa43100, transno 4295070166 00000004:00080000:2.0:1785220581.939794:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c0e00, transno 4295070167 00000004:00080000:2.0:1785220581.939797:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a262300, transno 4295070168 00000004:00080000:2.0:1785220581.939799:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a263800, transno 4295070169 00000004:00080000:2.0:1785220581.939802:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a69f80, transno 4295070170 00000004:00080000:2.0:1785220581.939804:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a260380, transno 4295070171 00000004:00080000:2.0:1785220581.939806:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a262d80, transno 4295070172 00000004:00080000:2.0:1785220581.939809:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a261500, transno 4295070173 00000004:00080000:2.0:1785220581.939811:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86311500, transno 4295070174 00000004:00080000:2.0:1785220581.939814:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f9880, transno 4295070175 00000004:00080000:2.0:1785220581.939816:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906fad80, transno 4295070176 00000004:00080000:2.0:1785220581.939818:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906fb100, transno 4295070177 00000004:00080000:2.0:1785220581.939820:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906fb800, transno 4295070178 00000100:00100000:1.0:1785220581.940458:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0000_UUID:14145:1871938510703744:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.940480:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.940503:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.940733:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.940775:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.940790:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960192 00000100:00100000:1.0:1785220581.940796:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960192 00010000:00010000:1.0:1785220581.940819:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007b5aac2b/0xf62499ab9278fa0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f283 expref: 207 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.940844:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960320 00000100:00100000:1.0:1785220581.940847:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960320 00010000:00010000:1.0:1785220581.940859:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b8afc1a7/0xf62499ab9278f67e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f22f expref: 208 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.940880:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100415 00000100:00100000:1.0:1785220581.940887:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+208:2407:x1871938519960192:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.940899:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.940904:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007b5aac2b/0xf62499ab9278fa0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f283 expref: 208 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.940915:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007b5aac2b/0xf62499ab9278fa0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f283 expref: 208 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.940935:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b5aac2b/0xf62499ab9278fa0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f283 expref: 208 pid: 14150 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.940945:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90409c00, transno 4295070179 00000004:00080000:2.0:1785220581.940947:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f9500, transno 4295070180 00010000:00010000:1.0:1785220581.940949:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729e:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.940950:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906fa300, transno 4295070181 00000004:00080000:2.0:1785220581.940952:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59f480, transno 4295070182 00010000:00010000:1.0:1785220581.940953:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729e:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.940955:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59ce00, transno 4295070183 00000004:00080000:2.0:1785220581.940957:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59df80, transno 4295070184 00010000:00010000:1.0:1785220581.940959:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.940960:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544b800, transno 4295070185 00000004:00080000:2.0:1785220581.940962:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a260000, transno 4295070186 00000004:00080000:2.0:1785220581.940965:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa42a00, transno 4295070187 00000004:00080000:2.0:1785220581.940967:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906fa680, transno 4295070188 00010000:00010000:1.0:1785220581.940967:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.940970:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa40700, transno 4295070189 00000004:00080000:2.0:1785220581.940972:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a261180, transno 4295070190 00000004:00080000:2.0:1785220581.940975:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a261f80, transno 4295070191 00000004:00080000:2.0:1785220581.940977:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a705c00, transno 4295070192 00000004:00080000:2.0:1785220581.940979:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a707b80, transno 4295070193 00000004:00080000:2.0:1785220581.940982:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add5f80, transno 4295070194 00000004:00080000:2.0:1785220581.940984:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f0a80, transno 4295070195 00010000:00010000:1.0:1785220581.940986:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220581.940987:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a706d80, transno 4295070196 00000004:00080000:2.0:1785220581.940989:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a704a80, transno 4295070197 00000004:00080000:2.0:1785220581.940992:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f2a00, transno 4295070198 00000004:00080000:2.0:1785220581.940994:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f3480, transno 4295070199 00000004:00080000:2.0:1785220581.940996:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a706680, transno 4295070200 00000004:00080000:2.0:1785220581.940999:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae72680, transno 4295070201 00000004:00080000:2.0:1785220581.941001:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae73b80, transno 4295070202 00000004:00080000:2.0:1785220581.941003:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae73100, transno 4295070203 00000004:00080000:2.0:1785220581.941006:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae71180, transno 4295070204 00000004:00080000:2.0:1785220581.941009:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97984700, transno 4295070205 00000004:00080000:2.0:1785220581.941011:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97985f80, transno 4295070206 00000004:00080000:2.0:1785220581.941013:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae71500, transno 4295070207 00000004:00080000:2.0:1785220581.941016:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae70380, transno 4295070208 00000100:00100000:1.0:1785220581.941016:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+207:2407:x1871938519960192:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 130us (284us total) trans 0 rc 0/0 00000004:00080000:2.0:1785220581.941018:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97985180, transno 4295070209 00000004:00080000:2.0:1785220581.941020:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97985880, transno 4295070210 00000100:00100000:1.0:1785220581.941022:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100415 00000004:00080000:2.0:1785220581.941023:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97985c00, transno 4295070211 00000004:00080000:2.0:1785220581.941025:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae72a00, transno 4295070212 00000004:00080000:2.0:1785220581.941028:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97987480, transno 4295070213 00000004:00080000:2.0:1785220581.941031:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69bb80, transno 4295070214 00000004:00080000:2.0:1785220581.941033:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69b480, transno 4295070215 00000004:00080000:2.0:1785220581.941035:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d55c00, transno 4295070216 00000100:00100000:1.0:1785220581.941037:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100416 00000004:00080000:2.0:1785220581.941038:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d57100, transno 4295070217 00000004:00080000:2.0:1785220581.941040:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69aa00, transno 4295070218 00000100:00100000:1.0:1785220581.941042:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+206:2407:x1871938519960320:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000004:00080000:2.0:1785220581.941043:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8c700, transno 4295070219 00000004:00080000:2.0:1785220581.941046:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8e680, transno 4295070220 00000004:00080000:2.0:1785220581.941048:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8f480, transno 4295070221 00010000:00010000:1.0:1785220581.941049:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:2.0:1785220581.941051:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8c380, transno 4295070222 00000004:00080000:2.0:1785220581.941053:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8ca80, transno 4295070223 00010000:00010000:1.0:1785220581.941054:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b8afc1a7/0xf62499ab9278f67e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f22f expref: 206 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.941056:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8dc00, transno 4295070224 00000004:00080000:2.0:1785220581.941058:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8c000, transno 4295070225 00000004:00080000:2.0:1785220581.941061:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493aa00, transno 4295070226 00010000:00010000:1.0:1785220581.941063:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b8afc1a7/0xf62499ab9278f67e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f22f expref: 206 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.941064:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94938e00, transno 4295070227 00000004:00080000:2.0:1785220581.941066:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94939c00, transno 4295070228 00000004:00080000:2.0:1785220581.941068:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94938000, transno 4295070229 00000004:00080000:2.0:1785220581.941071:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493bb80, transno 4295070230 00000004:00080000:2.0:1785220581.941074:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a98380, transno 4295070231 00010000:00010000:1.0:1785220581.941074:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7298:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.941076:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae70700, transno 4295070232 00010000:00010000:1.0:1785220581.941077:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7298:0x0].0x0 since=0s v=(1 0) 00000004:00080000:2.0:1785220581.941079:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9bb80, transno 4295070233 00010000:00010000:1.0:1785220581.941081:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.941082:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a98000, transno 4295070234 00010000:00010000:1.0:1785220581.941112:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.941126:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.941144:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+206:2407:x1871938519960320:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (368us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.941150:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100416 00010000:00010000:1.0:1785220581.941173:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.941182:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.941191:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.941192:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a98a80, transno 4295070235 00000004:00080000:2.0:1785220581.941195:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae73800, transno 4295070236 00000004:00080000:2.0:1785220581.941197:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae72300, transno 4295070237 00010000:00010000:1.0:1785220581.941199:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220581.941200:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae70a80, transno 4295070238 00000004:00080000:2.0:1785220581.941202:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae72d80, transno 4295070239 00000004:00080000:2.0:1785220581.941204:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a99880, transno 4295070240 00000004:00080000:2.0:1785220581.941207:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a99f80, transno 4295070241 00000004:00080000:2.0:1785220581.941209:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6b100, transno 4295070242 00000004:00080000:2.0:1785220581.941212:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493b480, transno 4295070243 00000004:00080000:2.0:1785220581.941215:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b3b80, transno 4295070244 00010000:00010000:1.0:1785220581.941215:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1785220581.941217:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493ad80, transno 4295070245 00000004:00080000:2.0:1785220581.941219:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94939880, transno 4295070246 00000004:00080000:2.0:1785220581.941221:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94938a80, transno 4295070247 00000004:00080000:2.0:1785220581.941224:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f9f80, transno 4295070248 00000004:00080000:2.0:1785220581.941226:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f8380, transno 4295070249 00000004:00080000:2.0:1785220581.941228:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f8000, transno 4295070250 00000004:00080000:2.0:1785220581.941231:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adc700, transno 4295070251 00000004:00080000:2.0:1785220581.941233:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d699880, transno 4295070252 00000004:00080000:2.0:1785220581.941236:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a704380, transno 4295070253 00000004:00080000:2.0:1785220581.941238:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f8e00, transno 4295070254 00000004:00080000:2.0:1785220581.941240:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae71c00, transno 4295070255 00000004:00080000:2.0:1785220581.941243:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f2680, transno 4295070256 00000004:00080000:2.0:1785220581.941245:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8aed80, transno 4295070257 00000004:00080000:2.0:1785220581.941248:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97986300, transno 4295070258 00000004:00080000:2.0:1785220581.941250:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8adc00, transno 4295070259 00000004:00080000:2.0:1785220581.941252:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ad500, transno 4295070260 00000004:00080000:2.0:1785220581.941254:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f3800, transno 4295070261 00000004:00080000:2.0:1785220581.941257:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ace00, transno 4295070262 00000004:00080000:2.0:1785220581.941259:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937a680, transno 4295070263 00000004:00080000:2.0:1785220581.941261:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ac380, transno 4295070264 00000004:00080000:2.0:1785220581.941264:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97984e00, transno 4295070265 00000004:00080000:2.0:1785220581.941266:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8aea00, transno 4295070266 00000004:00080000:2.0:1785220581.941269:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8af480, transno 4295070267 00000004:00080000:2.0:1785220581.941272:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ae680, transno 4295070268 00000004:00080000:2.0:1785220581.941274:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ac700, transno 4295070269 00000004:00080000:2.0:1785220581.941276:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c86a00, transno 4295070270 00000004:00080000:2.0:1785220581.941278:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97840380, transno 4295070271 00000004:00080000:2.0:1785220581.941281:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c84380, transno 4295070272 00000004:00080000:2.0:1785220581.941283:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97841180, transno 4295070273 00000004:00080000:2.0:1785220581.941285:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97841c00, transno 4295070274 00000004:00080000:2.0:1785220581.941287:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f8000, transno 4295070275 00000004:00080000:2.0:1785220581.941290:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97843480, transno 4295070276 00000004:00080000:2.0:1785220581.941292:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fa680, transno 4295070277 00000004:00080000:2.0:1785220581.941294:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97840700, transno 4295070278 00000004:00080000:2.0:1785220581.941296:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f9c00, transno 4295070279 00000004:00080000:2.0:1785220581.941298:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f9880, transno 4295070280 00000004:00080000:2.0:1785220581.941301:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f9f80, transno 4295070281 00000004:00080000:2.0:1785220581.941303:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921faa00, transno 4295070282 00000004:00080000:2.0:1785220581.941305:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f8700, transno 4295070283 00000004:00080000:2.0:1785220581.941307:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c86d80, transno 4295070284 00000004:00080000:2.0:1785220581.941310:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a705880, transno 4295070285 00000004:00080000:2.0:1785220581.941312:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c85500, transno 4295070286 00000004:00080000:2.0:1785220581.941314:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c87800, transno 4295070287 00000004:00080000:2.0:1785220581.941316:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c85f80, transno 4295070288 00000004:00080000:2.0:1785220581.941318:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c86680, transno 4295070289 00000004:00080000:2.0:1785220581.941321:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add5500, transno 4295070290 00000004:00080000:2.0:1785220581.941323:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8f800, transno 4295070291 00000004:00080000:2.0:1785220581.941326:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8ea00, transno 4295070292 00000004:00080000:2.0:1785220581.941328:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8df80, transno 4295070293 00000004:00080000:2.0:1785220581.941331:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8ed80, transno 4295070294 00000004:00080000:2.0:1785220581.941333:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a707800, transno 4295070295 00000004:00080000:2.0:1785220581.941336:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a705180, transno 4295070296 00000004:00080000:2.0:1785220581.941338:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a98e00, transno 4295070297 00000004:00080000:2.0:1785220581.941341:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a263b80, transno 4295070298 00000004:00080000:2.0:1785220581.941343:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d311c00, transno 4295070299 00000004:00080000:2.0:1785220581.941346:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae93100, transno 4295070300 00000004:00080000:2.0:1785220581.941347:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c84700, transno 4295070301 00000004:00080000:2.0:1785220581.941350:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c84a80, transno 4295070302 00000004:00080000:2.0:1785220581.941352:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d312d80, transno 4295070303 00000100:00100000:1.0:1785220581.941353:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+21:3655:x1871938510703616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9333us (15016us total) trans 4295070647 rc 0/0 00000004:00080000:2.0:1785220581.941355:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a263100, transno 4295070304 00000004:00080000:2.0:1785220581.941358:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9aa00, transno 4295070305 00000004:00080000:2.0:1785220581.941360:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9ad80, transno 4295070306 00000100:00100000:1.0:1785220581.941361:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284482 00000004:00080000:2.0:1785220581.941362:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9b800, transno 4295070307 00000004:00080000:2.0:1785220581.941365:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a99c00, transno 4295070308 00000004:00080000:2.0:1785220581.941367:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97841500, transno 4295070309 00000004:00080000:2.0:1785220581.941370:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c681f80, transno 4295070310 00000004:00080000:2.0:1785220581.941372:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d698700, transno 4295070311 00000004:00080000:2.0:1785220581.941374:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ade680, transno 4295070312 00000004:00080000:2.0:1785220581.941377:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98addc00, transno 4295070313 00000004:00080000:2.0:1785220581.941380:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8afb80, transno 4295070314 00000004:00080000:2.0:1785220581.941382:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad0380, transno 4295070315 00000004:00080000:2.0:1785220581.941384:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c682680, transno 4295070316 00000004:00080000:2.0:1785220581.941386:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d699500, transno 4295070317 00000004:00080000:2.0:1785220581.941388:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906faa00, transno 4295070318 00000004:00080000:2.0:1785220581.941391:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adce00, transno 4295070319 00000004:00080000:2.0:1785220581.941393:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985ebb80, transno 4295070320 00000004:00080000:2.0:1785220581.941395:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f9180, transno 4295070321 00000004:00080000:2.0:1785220581.941397:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c84000, transno 4295070322 00000004:00080000:2.0:1785220581.941400:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d54700, transno 4295070323 00000004:00080000:2.0:1785220581.941402:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c84e00, transno 4295070324 00000004:00080000:2.0:1785220581.941405:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d55f80, transno 4295070325 00000004:00080000:2.0:1785220581.941407:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d54a80, transno 4295070326 00000004:00080000:2.0:1785220581.941410:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c85c00, transno 4295070327 00000004:00080000:2.0:1785220581.941412:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998b9880, transno 4295070328 00000004:00080000:2.0:1785220581.941414:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b008a80, transno 4295070329 00000004:00080000:2.0:1785220581.941416:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00a680, transno 4295070330 00000004:00080000:2.0:1785220581.941418:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a260e00, transno 4295070331 00000004:00080000:2.0:1785220581.941420:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b008700, transno 4295070332 00000004:00080000:2.0:1785220581.941423:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906fb480, transno 4295070333 00000004:00080000:2.0:1785220581.941426:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ade300, transno 4295070334 00000004:00080000:2.0:1785220581.941428:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00bb80, transno 4295070335 00000004:00080000:2.0:1785220581.941431:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4d880, transno 4295070336 00000004:00080000:2.0:1785220581.941433:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adc000, transno 4295070337 00000004:00080000:2.0:1785220581.941436:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90141500, transno 4295070338 00000004:00080000:2.0:1785220581.941438:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90141880, transno 4295070339 00000004:00080000:2.0:1785220581.941440:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90140700, transno 4295070340 00000004:00080000:2.0:1785220581.941443:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a704e00, transno 4295070341 00000004:00080000:2.0:1785220581.941445:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90141180, transno 4295070342 00000004:00080000:2.0:1785220581.941448:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b031500, transno 4295070343 00000004:00080000:2.0:1785220581.941451:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b032300, transno 4295070344 00000004:00080000:2.0:1785220581.941453:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b031880, transno 4295070345 00000004:00080000:2.0:1785220581.941455:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998b8e00, transno 4295070346 00000004:00080000:2.0:1785220581.941458:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998b8380, transno 4295070347 00000004:00080000:2.0:1785220581.941460:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998b9180, transno 4295070348 00000004:00080000:2.0:1785220581.941463:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998bb480, transno 4295070349 00000004:00080000:2.0:1785220581.941467:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa60a80, transno 4295070350 00000004:00080000:2.0:1785220581.941469:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b032d80, transno 4295070351 00000004:00080000:2.0:1785220581.941471:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69a680, transno 4295070352 00000004:00080000:2.0:1785220581.941473:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69ad80, transno 4295070353 00000004:00080000:2.0:1785220581.941476:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa63800, transno 4295070354 00000004:00080000:2.0:1785220581.941478:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91478700, transno 4295070355 00000004:00080000:2.0:1785220581.941480:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147bb80, transno 4295070356 00000004:00080000:2.0:1785220581.941482:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147b100, transno 4295070357 00000004:00080000:2.0:1785220581.941484:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d310700, transno 4295070358 00000004:00080000:2.0:1785220581.941487:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91479180, transno 4295070359 00000004:00080000:2.0:1785220581.941489:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b009f80, transno 4295070360 00000004:00080000:2.0:1785220581.941491:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4f100, transno 4295070361 00000004:00080000:2.0:1785220581.941493:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d312300, transno 4295070362 00000004:00080000:2.0:1785220581.941496:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5aa00, transno 4295070363 00000004:00080000:2.0:1785220581.941498:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4fb80, transno 4295070364 00000004:00080000:2.0:1785220581.941500:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d312680, transno 4295070365 00000004:00080000:2.0:1785220581.941503:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4dc00, transno 4295070366 00000004:00080000:2.0:1785220581.941505:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4df80, transno 4295070367 00000004:00080000:2.0:1785220581.941508:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915af800, transno 4295070368 00000004:00080000:2.0:1785220581.941510:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360b800, transno 4295070369 00000004:00080000:2.0:1785220581.941513:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ae300, transno 4295070370 00000004:00080000:2.0:1785220581.941515:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96666a00, transno 4295070371 00000004:00080000:2.0:1785220581.941518:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761f80, transno 4295070372 00000004:00080000:2.0:1785220581.941520:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4e680, transno 4295070373 00000004:00080000:2.0:1785220581.941523:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360b100, transno 4295070374 00000004:00080000:2.0:1785220581.941525:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fb800, transno 4295070375 00000004:00080000:2.0:1785220581.941528:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adca80, transno 4295070376 00000004:00080000:2.0:1785220581.941530:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4e300, transno 4295070377 00000004:00080000:2.0:1785220581.941532:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4d180, transno 4295070378 00000004:00080000:2.0:1785220581.941535:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147b480, transno 4295070379 00000004:00080000:2.0:1785220581.941537:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360aa00, transno 4295070380 00000004:00080000:2.0:1785220581.941540:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72df80, transno 4295070381 00000004:00080000:2.0:1785220581.941542:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c682d80, transno 4295070382 00000004:00080000:2.0:1785220581.941545:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90140380, transno 4295070383 00000004:00080000:2.0:1785220581.941547:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ac700, transno 4295070384 00000004:00080000:2.0:1785220581.941549:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72d180, transno 4295070385 00000004:00080000:2.0:1785220581.941552:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72f800, transno 4295070386 00000004:00080000:2.0:1785220581.941554:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72f480, transno 4295070387 00000004:00080000:2.0:1785220581.941557:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93609180, transno 4295070388 00000004:00080000:2.0:1785220581.941559:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b008380, transno 4295070389 00000004:00080000:2.0:1785220581.941561:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b008e00, transno 4295070390 00000004:00080000:2.0:1785220581.941564:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbe300, transno 4295070391 00000004:00080000:2.0:1785220581.941566:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147a680, transno 4295070392 00000004:00080000:2.0:1785220581.941568:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbd880, transno 4295070393 00000004:00080000:2.0:1785220581.941571:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c928a80, transno 4295070394 00000004:00080000:2.0:1785220581.941573:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a4700, transno 4295070395 00000004:00080000:2.0:1785220581.941576:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a7b80, transno 4295070396 00000004:00080000:2.0:1785220581.941577:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a4380, transno 4295070397 00000004:00080000:2.0:1785220581.941580:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a6d80, transno 4295070398 00000004:00080000:2.0:1785220581.941582:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbf800, transno 4295070399 00000004:00080000:2.0:1785220581.941583:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbe680, transno 4295070400 00000004:00080000:2.0:1785220581.941586:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93608380, transno 4295070401 00000004:00080000:2.0:1785220581.941588:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360b480, transno 4295070402 00000004:00080000:2.0:1785220581.941590:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a4e00, transno 4295070403 00000004:00080000:2.0:1785220581.941593:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365c000, transno 4295070404 00000004:00080000:2.0:1785220581.941595:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365ce00, transno 4295070405 00000004:00080000:2.0:1785220581.941598:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365ed80, transno 4295070406 00000004:00080000:2.0:1785220581.941600:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365e300, transno 4295070407 00000004:00080000:2.0:1785220581.941602:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365f800, transno 4295070408 00000004:00080000:2.0:1785220581.941605:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93609500, transno 4295070409 00000004:00080000:2.0:1785220581.941607:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbfb80, transno 4295070410 00000004:00080000:2.0:1785220581.941609:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbca80, transno 4295070411 00000004:00080000:2.0:1785220581.941611:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93664380, transno 4295070412 00000004:00080000:2.0:1785220581.941613:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbf480, transno 4295070413 00000004:00080000:2.0:1785220581.941616:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbdc00, transno 4295070414 00000004:00080000:2.0:1785220581.941618:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbd880, transno 4295070415 00000004:00080000:2.0:1785220581.941620:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93609f80, transno 4295070416 00000004:00080000:2.0:1785220581.941623:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93664e00, transno 4295070417 00000004:00080000:2.0:1785220581.941625:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97842300, transno 4295070418 00000004:00080000:2.0:1785220581.941628:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b030a80, transno 4295070419 00000004:00080000:2.0:1785220581.941630:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97840000, transno 4295070420 00000004:00080000:2.0:1785220581.941633:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b031180, transno 4295070421 00000004:00080000:2.0:1785220581.941635:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b033b80, transno 4295070422 00000004:00080000:2.0:1785220581.941639:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97840e00, transno 4295070423 00000004:00080000:2.0:1785220581.941641:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9b480, transno 4295070424 00000004:00080000:2.0:1785220581.941643:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba5c00, transno 4295070425 00000004:00080000:2.0:1785220581.941646:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba5f80, transno 4295070426 00000004:00080000:2.0:1785220581.941649:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba7100, transno 4295070427 00000004:00080000:2.0:1785220581.941651:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba4380, transno 4295070428 00000004:00080000:2.0:1785220581.941653:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba6d80, transno 4295070429 00000004:00080000:2.0:1785220581.941655:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba4700, transno 4295070430 00000004:00080000:2.0:1785220581.941658:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba7b80, transno 4295070431 00000004:00080000:2.0:1785220581.941660:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba5180, transno 4295070432 00000004:00080000:2.0:1785220581.941662:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa2300, transno 4295070433 00000004:00080000:2.0:1785220581.941665:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa3b80, transno 4295070434 00000004:00080000:2.0:1785220581.941667:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa0a80, transno 4295070435 00000004:00080000:2.0:1785220581.941670:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fdc00, transno 4295070436 00000004:00080000:2.0:1785220581.941672:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955ff800, transno 4295070437 00000004:00080000:2.0:1785220581.941675:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fce00, transno 4295070438 00000004:00080000:2.0:1785220581.941678:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fca80, transno 4295070439 00000004:00080000:2.0:1785220581.941680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99919c00, transno 4295070440 00000004:00080000:2.0:1785220581.941682:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10ad80, transno 4295070441 00000004:00080000:2.0:1785220581.941685:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10b100, transno 4295070442 00000004:00080000:2.0:1785220581.941687:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10b800, transno 4295070443 00000004:00080000:2.0:1785220581.941689:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b108000, transno 4295070444 00000004:00080000:2.0:1785220581.941691:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10b480, transno 4295070445 00000004:00080000:2.0:1785220581.941693:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbf100, transno 4295070446 00000004:00080000:2.0:1785220581.941695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbe300, transno 4295070447 00000004:00080000:2.0:1785220581.941697:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbea00, transno 4295070448 00000004:00080000:2.0:1785220581.941699:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97841880, transno 4295070449 00000004:00080000:2.0:1785220581.941701:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6eca80, transno 4295070450 00000004:00080000:2.0:1785220581.941703:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbf800, transno 4295070451 00000004:00080000:2.0:1785220581.941705:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbdf80, transno 4295070452 00000004:00080000:2.0:1785220581.941708:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915afb80, transno 4295070453 00000004:00080000:2.0:1785220581.941710:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97843b80, transno 4295070454 00000004:00080000:2.0:1785220581.941712:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365d500, transno 4295070455 00000004:00080000:2.0:1785220581.941715:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa62d80, transno 4295070456 00000004:00080000:2.0:1785220581.941717:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ad500, transno 4295070457 00000004:00080000:2.0:1785220581.941719:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97842a00, transno 4295070458 00000004:00080000:2.0:1785220581.941721:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b031f80, transno 4295070459 00000004:00080000:2.0:1785220581.941723:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa61880, transno 4295070460 00000004:00080000:2.0:1785220581.941725:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c681880, transno 4295070461 00000004:00080000:2.0:1785220581.941727:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa60e00, transno 4295070462 00000004:00080000:2.0:1785220581.941729:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b030700, transno 4295070463 00000004:00080000:2.0:1785220581.941732:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72c000, transno 4295070464 00000004:00080000:2.0:1785220581.941734:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72e680, transno 4295070465 00000004:00080000:2.0:1785220581.941736:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72ea00, transno 4295070466 00000004:00080000:2.0:1785220581.941738:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72f100, transno 4295070467 00000004:00080000:2.0:1785220581.941740:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72fb80, transno 4295070468 00000004:00080000:2.0:1785220581.941742:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72d880, transno 4295070469 00000004:00080000:2.0:1785220581.941744:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72ca80, transno 4295070470 00000004:00080000:2.0:1785220581.941746:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f724a80, transno 4295070471 00000004:00080000:2.0:1785220581.941748:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be4000, transno 4295070472 00000004:00080000:2.0:1785220581.941750:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f724700, transno 4295070473 00000004:00080000:2.0:1785220581.941752:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be4a80, transno 4295070474 00010000:00010000:1.0:1785220581.942245:0:13874:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8afc1a7/0xf62499ab9278f67e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f22f expref: 205 pid: 13819 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.942258:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510703872:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.942276:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.942285:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.942291:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.942299:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.942306:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.942321:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.942427:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+21:3652:x1871938510700928:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9908us (16354us total) trans 4295070648 rc 0/0 00000100:00100000:1.0:1785220581.942433:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284483 00000004:00080000:2.0:1785220581.944258:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915aca80, transno 4295070475 00000004:00080000:2.0:1785220581.944266:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be7100, transno 4295070476 00000004:00080000:2.0:1785220581.944269:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be5180, transno 4295070477 00000004:00080000:2.0:1785220581.944272:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f725f80, transno 4295070478 00000004:00080000:2.0:1785220581.944274:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be7480, transno 4295070479 00000004:00080000:2.0:1785220581.944276:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f724e00, transno 4295070480 00000004:00080000:2.0:1785220581.944279:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4c380, transno 4295070481 00000004:00080000:2.0:1785220581.944285:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ac000, transno 4295070482 00000004:00080000:2.0:1785220581.944289:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb2d80, transno 4295070483 00000004:00080000:2.0:1785220581.944292:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998b9500, transno 4295070484 00000004:00080000:2.0:1785220581.944295:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72ed80, transno 4295070485 00000004:00080000:2.0:1785220581.944297:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4ed80, transno 4295070486 00000004:00080000:2.0:1785220581.944299:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be4e00, transno 4295070487 00000004:00080000:2.0:1785220581.944302:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e04380, transno 4295070488 00000004:00080000:2.0:1785220581.944304:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e07b80, transno 4295070489 00000004:00080000:2.0:1785220581.944310:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93638700, transno 4295070490 00000004:00080000:2.0:1785220581.944313:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbc700, transno 4295070491 00000004:00080000:2.0:1785220581.944315:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e07480, transno 4295070492 00000004:00080000:2.0:1785220581.944320:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbf480, transno 4295070493 00000004:00080000:2.0:1785220581.944323:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbd180, transno 4295070494 00000004:00080000:2.0:1785220581.944325:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72d500, transno 4295070495 00000004:00080000:2.0:1785220581.944327:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f726300, transno 4295070496 00000004:00080000:2.0:1785220581.944333:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b030000, transno 4295070497 00000004:00080000:2.0:1785220581.944335:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f724000, transno 4295070498 00000004:00080000:2.0:1785220581.944337:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72c700, transno 4295070499 00000004:00080000:2.0:1785220581.944339:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f725180, transno 4295070500 00000004:00080000:2.0:1785220581.944344:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a5880, transno 4295070501 00000004:00080000:2.0:1785220581.944347:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72e300, transno 4295070502 00000004:00080000:2.0:1785220581.944349:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be5f80, transno 4295070503 00000004:00080000:2.0:1785220581.944353:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ad880, transno 4295070504 00000004:00080000:2.0:1785220581.944355:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f726680, transno 4295070505 00000004:00080000:2.0:1785220581.944360:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f724380, transno 4295070506 00000004:00080000:2.0:1785220581.944363:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998bbb80, transno 4295070507 00000004:00080000:2.0:1785220581.944365:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e04700, transno 4295070508 00000004:00080000:2.0:1785220581.944367:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72c380, transno 4295070509 00000004:00080000:2.0:1785220581.944372:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbdf80, transno 4295070510 00000004:00080000:2.0:1785220581.944375:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e04000, transno 4295070511 00000004:00080000:2.0:1785220581.944377:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbea00, transno 4295070512 00000004:00080000:2.0:1785220581.944382:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e06a00, transno 4295070513 00000004:00080000:2.0:1785220581.944385:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f725880, transno 4295070514 00000004:00080000:2.0:1785220581.944387:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbce00, transno 4295070515 00000004:00080000:2.0:1785220581.944390:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91479880, transno 4295070516 00000004:00080000:2.0:1785220581.944394:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbdc00, transno 4295070517 00000004:00080000:2.0:1785220581.944397:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ed880, transno 4295070518 00000004:00080000:2.0:1785220581.944399:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ec700, transno 4295070519 00000004:00080000:2.0:1785220581.944404:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ee680, transno 4295070520 00000004:00080000:2.0:1785220581.944407:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99919180, transno 4295070521 00000004:00080000:2.0:1785220581.944409:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ef800, transno 4295070522 00000004:00080000:2.0:1785220581.944412:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ef100, transno 4295070523 00000004:00080000:2.0:1785220581.944416:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99918380, transno 4295070524 00000004:00080000:2.0:1785220581.944418:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f725c00, transno 4295070525 00000004:00080000:2.0:1785220581.944421:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb20e00, transno 4295070526 00000004:00080000:2.0:1785220581.944423:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e05500, transno 4295070527 00000004:00080000:2.0:1785220581.944428:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb22680, transno 4295070528 00000004:00080000:2.0:1785220581.944430:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f726a00, transno 4295070529 00000004:00080000:2.0:1785220581.944433:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5aa00, transno 4295070530 00000004:00080000:2.0:1785220581.944435:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99918a80, transno 4295070531 00000004:00080000:2.0:1785220581.944440:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb21880, transno 4295070532 00000004:00080000:2.0:1785220581.944443:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb67b80, transno 4295070533 00000004:00080000:2.0:1785220581.944445:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb64380, transno 4295070534 00000004:00080000:2.0:1785220581.944447:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb64e00, transno 4295070535 00000004:00080000:2.0:1785220581.944452:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb64000, transno 4295070536 00000004:00080000:2.0:1785220581.944454:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb67800, transno 4295070537 00000004:00080000:2.0:1785220581.944456:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb65f80, transno 4295070538 00000004:00080000:2.0:1785220581.944458:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb66300, transno 4295070539 00000004:00080000:2.0:1785220581.944460:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601d880, transno 4295070540 00000004:00080000:2.0:1785220581.944464:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601ce00, transno 4295070541 00000004:00080000:2.0:1785220581.944467:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601e300, transno 4295070542 00000004:00080000:2.0:1785220581.944470:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601d500, transno 4295070543 00000004:00080000:2.0:1785220581.944472:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601c380, transno 4295070544 00000004:00080000:2.0:1785220581.944474:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601d180, transno 4295070545 00000004:00080000:2.0:1785220581.944479:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601e680, transno 4295070546 00000004:00080000:2.0:1785220581.944482:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601f800, transno 4295070547 00000004:00080000:2.0:1785220581.944484:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601ed80, transno 4295070548 00000004:00080000:2.0:1785220581.944486:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fd180, transno 4295070549 00000004:00080000:2.0:1785220581.944490:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99918000, transno 4295070550 00000004:00080000:2.0:1785220581.944493:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991bb80, transno 4295070551 00000004:00080000:2.0:1785220581.944495:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93609c00, transno 4295070552 00000004:00080000:2.0:1785220581.944497:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99918e00, transno 4295070553 00000004:00080000:2.0:1785220581.944502:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb20000, transno 4295070554 00000004:00080000:2.0:1785220581.944505:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb23480, transno 4295070555 00000004:00080000:2.0:1785220581.944507:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fe680, transno 4295070556 00000004:00080000:2.0:1785220581.944509:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93664700, transno 4295070557 00000004:00080000:2.0:1785220581.944514:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fc000, transno 4295070558 00000004:00080000:2.0:1785220581.944516:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93665500, transno 4295070559 00000004:00080000:2.0:1785220581.944519:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601fb80, transno 4295070560 00000004:00080000:2.0:1785220581.944524:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fdf80, transno 4295070561 00000004:00080000:2.0:1785220581.944526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb20700, transno 4295070562 00000004:00080000:2.0:1785220581.944529:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb23b80, transno 4295070563 00000004:00080000:2.0:1785220581.944531:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb22d80, transno 4295070564 00000004:00080000:2.0:1785220581.944534:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbed80, transno 4295070565 00000004:00080000:2.0:1785220581.944539:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7e680, transno 4295070566 00000004:00080000:2.0:1785220581.944541:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb67100, transno 4295070567 00000004:00080000:2.0:1785220581.944544:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90143480, transno 4295070568 00000004:00080000:2.0:1785220581.944549:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb21500, transno 4295070569 00000004:00080000:2.0:1785220581.944551:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbe680, transno 4295070570 00000004:00080000:2.0:1785220581.944554:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7dc00, transno 4295070571 00000004:00080000:2.0:1785220581.944556:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7ca80, transno 4295070572 00000004:00080000:2.0:1785220581.944559:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083ce00, transno 4295070573 00000004:00080000:2.0:1785220581.944561:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7d500, transno 4295070574 00000004:00080000:2.0:1785220581.944566:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7f800, transno 4295070575 00000004:00080000:2.0:1785220581.944568:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7fb80, transno 4295070576 00000004:00080000:2.0:1785220581.944570:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b033100, transno 4295070577 00000004:00080000:2.0:1785220581.944572:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7f100, transno 4295070578 00000004:00080000:2.0:1785220581.944575:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba6300, transno 4295070579 00000004:00080000:2.0:1785220581.944589:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba4a80, transno 4295070580 00000004:00080000:2.0:1785220581.944591:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b49180, transno 4295070581 00000004:00080000:2.0:1785220581.944593:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba6a00, transno 4295070582 00000004:00080000:2.0:1785220581.944625:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b49500, transno 4295070583 00000004:00080000:2.0:1785220581.944627:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083ea00, transno 4295070584 00000004:00080000:2.0:1785220581.944654:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083f480, transno 4295070585 00000004:00080000:2.0:1785220581.944669:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083d880, transno 4295070586 00000004:00080000:2.0:1785220581.944684:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083c380, transno 4295070587 00000004:00080000:2.0:1785220581.944698:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083e680, transno 4295070588 00000004:00080000:2.0:1785220581.944724:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4b100, transno 4295070589 00000004:00080000:2.0:1785220581.944746:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b48e00, transno 4295070590 00000004:00080000:2.0:1785220581.944749:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b49f80, transno 4295070591 00000004:00080000:2.0:1785220581.944751:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4a680, transno 4295070592 00000004:00080000:2.0:1785220581.944762:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b48000, transno 4295070593 00000004:00080000:2.0:1785220581.944765:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4bb80, transno 4295070594 00000004:00080000:2.0:1785220581.944768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b48380, transno 4295070595 00000004:00080000:2.0:1785220581.944770:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4aa00, transno 4295070596 00000004:00080000:2.0:1785220581.944773:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94912300, transno 4295070597 00000004:00080000:2.0:1785220581.944775:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be5880, transno 4295070598 00000004:00080000:2.0:1785220581.944777:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94911500, transno 4295070599 00000004:00080000:2.0:1785220581.944779:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94911c00, transno 4295070600 00000004:00080000:2.0:1785220581.944781:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94910e00, transno 4295070601 00000004:00080000:2.0:1785220581.944783:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991a300, transno 4295070602 00000004:00080000:2.0:1785220581.944787:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94910380, transno 4295070603 00000004:00080000:2.0:1785220581.944789:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99919f80, transno 4295070604 00000004:00080000:2.0:1785220581.944791:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7d180, transno 4295070605 00000004:00080000:2.0:1785220581.944794:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c0000, transno 4295070606 00000004:00080000:2.0:1785220581.944796:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c0e00, transno 4295070607 00000004:00080000:2.0:1785220581.944799:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b1ee300, transno 4295070608 00000004:00080000:2.0:1785220581.944801:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a85f80, transno 4295070609 00000004:00080000:2.0:1785220581.944803:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a84380, transno 4295070610 00000004:00080000:2.0:1785220581.944805:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a86680, transno 4295070611 00000004:00080000:2.0:1785220581.944808:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb0e00, transno 4295070612 00000004:00080000:2.0:1785220581.944811:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a84a80, transno 4295070613 00000004:00080000:2.0:1785220581.944813:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94911f80, transno 4295070614 00000004:00080000:2.0:1785220581.944815:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601dc00, transno 4295070615 00000004:00080000:2.0:1785220581.944817:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93664a80, transno 4295070616 00000004:00080000:2.0:1785220581.944819:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991b100, transno 4295070617 00000004:00080000:2.0:1785220581.945015:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fed80, transno 4295070618 00000004:00080000:2.0:1785220581.945048:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb22a00, transno 4295070619 00000004:00080000:2.0:1785220581.945064:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991a680, transno 4295070620 00000004:00080000:2.0:1785220581.945069:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac8a80, transno 4295070621 00000004:00080000:2.0:1785220581.945083:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fc380, transno 4295070622 00000004:00080000:2.0:1785220581.945207:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b1ec380, transno 4295070623 00000004:00080000:2.0:1785220581.945233:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac9c00, transno 4295070624 00000004:00080000:2.0:1785220581.945239:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac8e00, transno 4295070625 00000004:00080000:2.0:1785220581.945241:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083f800, transno 4295070626 00000004:00080000:2.0:1785220581.945286:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b1ed880, transno 4295070627 00000004:00080000:2.0:1785220581.945292:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94acb480, transno 4295070628 00000004:00080000:2.0:1785220581.945294:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac8380, transno 4295070629 00000004:00080000:2.0:1785220581.945320:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac8000, transno 4295070630 00000004:00080000:2.0:1785220581.945346:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0ca80, transno 4295070631 00000004:00080000:2.0:1785220581.945361:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89efc380, transno 4295070632 00000004:00080000:2.0:1785220581.945387:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89efe300, transno 4295070633 00000100:00100000:2.0:1785220581.945457:0:3655:0:(client.c:3156:ptlrpc_free_committed()) @@@ stopping search req@ffff960d89efd880 x1871938510695296/t4295070634(4295070634) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1785220597 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1785220581.945547:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ec000/2, rc 0, transno 4295070642 00000004:00080000:0.0:1785220581.945548:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ed180/2, rc 0, transno 4295070643 00000100:00100000:2.0:1785220581.945554:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510701312:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.945554:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510701696:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.945571:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:2.0:1785220581.945577:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510704256:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.945579:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945590:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.945594:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945605:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945623:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945630:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a51b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.945635:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.945654:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.945658:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945663:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.945666:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.945671:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:0.0:1785220581.945673:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.945676:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510704384:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.945683:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945690:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a51bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.945696:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.945706:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.945707:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.945709:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.945724:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510704256 00000100:00100000:0.0:1785220581.945731:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510704256 00000100:00100000:0.0:1785220581.945783:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510704384 00000100:00100000:0.0:1785220581.945787:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510704384 00000100:00100000:0.0:1785220581.945808:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284486 00000100:00100000:0.0:1785220581.945816:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1208:3654:x1871938510704256:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.945843:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29295 count 1 00010000:00010000:0.0:1785220581.945868:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945887:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000102d155e/0xf62499ab9278db65 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e547 expref: 244 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945896:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x726f:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.945901:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.945905:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945935:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000102d155e/0xf62499ab9278db65 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e547 expref: 244 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.945948:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000102d155e/0xf62499ab9278db65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e547 expref: 244 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.945965:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510704768:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.946021:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a4d5600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.946038:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284487 00000100:00100000:0.0:1785220581.946045:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1208:3654:x1871938510704384:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.946057:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29296 count 1 00010000:00010000:0.0:1785220581.946068:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.946080:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000071accfe9/0xf62499ab9278dc76 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5e1 expref: 244 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.946266:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:1.0:1785220581.946435:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1edc00/2, rc 0, transno 4295070648 00000100:00100000:1.0:1785220581.946441:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510700928:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.946473:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0001_UUID:14251:1871938510704512:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.946485:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.946511:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510704640:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.946519:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.946534:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960448 00000100:00100000:1.0:1785220581.946537:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960448 00010000:00010000:1.0:1785220581.946555:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e3d0afd1/0xf62499ab9278d990 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e47c expref: 245 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.946578:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100417 00000100:00100000:1.0:1785220581.946584:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+245:2407:x1871938519960448:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.946592:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.946597:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0001_UUID lock: 00000000e3d0afd1/0xf62499ab9278d990 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e47c expref: 245 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946607:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e3d0afd1/0xf62499ab9278d990 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e47c expref: 245 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946624:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e3d0afd1/0xf62499ab9278d990 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e47c expref: 245 pid: 14189 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946635:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726d:0x0].0x0 since=1s v=(1 0) 00010000:00010000:1.0:1785220581.946638:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.946643:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946651:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946667:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.946691:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+244:2407:x1871938519960448:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (425us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.946698:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100417 00010000:00010000:1.0:1785220581.946717:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946725:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946733:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946740:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946753:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.946909:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+1209:3652:x1871938510704000:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7794us (7901us total) trans 4295070063 rc 0/0 00000100:00100000:1.0:1785220581.946916:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284484 00010000:00010000:1.0:1785220581.946934:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.946940:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946948:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.946951:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946959:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946971:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946978:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a51c2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.946993:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.946995:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.946997:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.947013:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c2680/2, rc 0, transno 4295070063 00000100:00100000:1.0:1785220581.947024:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510704000:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.947082:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510704896:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.947131:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.947213:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7270:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.947217:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7270:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.947221:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.947245:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000071accfe9/0xf62499ab9278dc76 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5e1 expref: 243 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.947256:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000071accfe9/0xf62499ab9278dc76 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5e1 expref: 243 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.947267:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510705024:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.947303:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9aedcd80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.947317:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510704896 00000100:00100000:0.0:1785220581.947321:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510704896 00000100:00100000:0.0:1785220581.947355:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284488 00000100:00100000:0.0:1785220581.947362:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1209:3652:x1871938510704896:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.947375:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29297 count 1 00010000:00010000:0.0:1785220581.947387:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.947398:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004f9b1a14/0xf62499ab9278dd25 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e651 expref: 243 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.947404:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7271:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.947407:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7271:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.947410:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.947427:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004f9b1a14/0xf62499ab9278dd25 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e651 expref: 243 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.947435:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004f9b1a14/0xf62499ab9278dd25 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e651 expref: 243 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.947443:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510705152:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.947464:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.947479:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220581.947498:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10bb80/2, rc 0, transno 4295070060 00000100:00100000:0.0:1785220581.947507:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510700160:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.947533:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220581.947539:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510705280:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.947560:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.947620:0:14204:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510705280 00000100:00100000:1.0:1785220581.947624:0:14204:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510705280 00000100:00100000:1.0:1785220581.947650:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284489 00000100:00100000:1.0:1785220581.947656:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1210:3652:x1871938510705280:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.947666:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29298 count 1 00010000:00010000:1.0:1785220581.947677:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.947688:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000b91a239/0xf62499ab9278dea6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e6f9 expref: 243 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.947695:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7272:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.947697:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7272:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.947701:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.947716:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000b91a239/0xf62499ab9278dea6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e6f9 expref: 243 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.947725:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000b91a239/0xf62499ab9278dea6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e6f9 expref: 243 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.947734:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510705408:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.947765:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464cc0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.948288:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.948298:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.948307:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.948314:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.948331:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.948461:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+21:3653:x1871938510701824:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24800us (26641us total) trans 4295070649 rc 0/0 00000100:00100000:0.0:1785220581.948469:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284475 00000004:00080000:0.0:1785220581.948583:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1efb80/2, rc 0, transno 4295070649 00000100:00100000:0.0:1785220581.948588:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510701824:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220581.948660:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.948666:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.948674:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.948678:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.948686:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.948706:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.948712:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5183 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.948727:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.948729:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.948730:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.949022:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.949081:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960576 00000004:00080000:1.0:1785220581.949142:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ec700/2, rc 0, transno 4295070061 00000100:00100000:1.0:1785220581.949154:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510700416:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.949210:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb66a00/2, rc 0, transno 4295070647 00000100:00100000:1.0:1785220581.949215:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510703616:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.949224:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c1500/2, rc 0, transno 4295070062 00000100:00100000:1.0:1785220581.949231:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510702464:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220581.949257:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c2d80/2, rc 0, transno 4295070058 00000100:00100000:1.0:1785220581.949263:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510702080:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.949296:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510705792:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.949334:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220581.949372:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1eca80/2, rc 0, transno 4295070646 00000100:00100000:1.0:1785220581.949377:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510701056:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.949386:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ed500/2, rc 0, transno 4295070644 00000100:00100000:1.0:1785220581.949389:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510701952:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220581.949398:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0c380/2, rc 0, transno 4295070645 00000100:00100000:1.0:1785220581.949401:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510700672:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.949408:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510705664:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.949431:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.949453:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220581.949459:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510705536:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.949478:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.949551:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960576 00010000:00010000:0.0:1785220581.949579:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003b69da5c/0xf62499ab9278da9a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4f3 expref: 244 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.949619:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100418 00000100:00100000:0.0:1785220581.949627:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+244:2406:x1871938519960576:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.949639:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.949644:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0001_UUID lock: 000000003b69da5c/0xf62499ab9278da9a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4f3 expref: 244 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949654:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003b69da5c/0xf62499ab9278da9a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4f3 expref: 244 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949670:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003b69da5c/0xf62499ab9278da9a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e4f3 expref: 244 pid: 14161 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949682:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726e:0x0].0x0 since=1s v=(1 0) 00010000:00010000:0.0:1785220581.949686:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.949691:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949699:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949722:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.949752:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+243:2406:x1871938519960576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 125us (731us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.949770:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100418 00000100:00100000:0.0:1785220581.949792:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510705792 00000100:00100000:0.0:1785220581.949795:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510705792 00000100:00100000:0.0:1785220581.949814:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510705664 00000100:00100000:0.0:1785220581.949816:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510705664 00000100:00100000:0.0:1785220581.949830:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510705536 00000100:00100000:0.0:1785220581.949833:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510705536 00000100:00100000:0.0:1785220581.949849:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284490 00000100:00100000:0.0:1785220581.949855:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1213:3652:x1871938510705792:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.949872:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29301 count 1 00010000:00010000:0.0:1785220581.949890:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949904:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000013aac52b/0xf62499ab9278e2ce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8d5 expref: 242 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949911:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7275:0x0].0x0 since=24s v=(0 0) 00010000:00010000:0.0:1785220581.949913:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7275:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.949917:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949938:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000013aac52b/0xf62499ab9278e2ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8d5 expref: 242 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.949948:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000013aac52b/0xf62499ab9278e2ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8d5 expref: 242 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.949958:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510705920:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.949997:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.950011:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.950020:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.950029:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.950036:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.950051:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.950427:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284491 00000100:00100000:2.0:1785220581.950437:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1213:3655:x1871938510705664:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.950452:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29300 count 1 00010000:00010000:2.0:1785220581.950465:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.950476:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005efb3a7d/0xf62499ab9278e1d2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e87a expref: 242 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.950482:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7274:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.950485:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7274:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.950489:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.950509:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005efb3a7d/0xf62499ab9278e1d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e87a expref: 242 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.950519:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005efb3a7d/0xf62499ab9278e1d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e87a expref: 242 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.950529:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510706048:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.950563:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.951210:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284492 00000100:00100000:1.0:1785220581.951221:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1213:3655:x1871938510705536:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.951238:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29299 count 1 00010000:00010000:1.0:1785220581.951251:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.951263:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000980ff11b/0xf62499ab9278dfe8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e793 expref: 242 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.951270:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7273:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.951272:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7273:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.951276:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.951301:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000980ff11b/0xf62499ab9278dfe8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e793 expref: 242 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.951310:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000980ff11b/0xf62499ab9278dfe8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e793 expref: 242 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.951320:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0001_UUID:14192:1871938510706176:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.951354:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464780 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.951606:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1214:3652:x1871938510704128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12210us (12264us total) trans 4295070064 rc 0/0 00000100:00100000:0.0:1785220581.951614:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284485 00000100:00100000:0.0:1785220581.951658:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510704768:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.951671:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.951687:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.951692:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.951700:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.951705:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.951713:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.951729:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.951735:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a51c9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.951750:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.951752:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.951765:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.951853:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c1c00/2, rc 0, transno 4295070064 00000100:00100000:1.0:1785220581.951903:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510704128:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.951986:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510706304:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.952015:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.952075:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510706304 00000100:00100000:2.0:1785220581.952081:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510706304 00000100:00100000:2.0:1785220581.952136:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284493 00000100:00100000:2.0:1785220581.952143:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1214:3653:x1871938510706304:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.952156:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29302 count 1 00010000:00010000:2.0:1785220581.952169:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.952182:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cac16859/0xf62499ab9278e3fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e95a expref: 242 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.952188:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7276:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.952191:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7276:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.952195:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.952214:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cac16859/0xf62499ab9278e3fb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e95a expref: 242 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.952228:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cac16859/0xf62499ab9278e3fb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e95a expref: 242 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.952238:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510706432:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.952274:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.954291:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220581.954349:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960704 00000100:00100000:0.0:1785220581.954356:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960704 00010000:00010000:0.0:1785220581.954382:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000102d155e/0xf62499ab9278db65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e547 expref: 243 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.954417:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100419 00000100:00100000:0.0:1785220581.954425:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+243:2407:x1871938519960704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.954437:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.954441:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0001_UUID lock: 00000000102d155e/0xf62499ab9278db65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e547 expref: 243 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.954452:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000102d155e/0xf62499ab9278db65 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e547 expref: 243 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.954474:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000102d155e/0xf62499ab9278db65 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e547 expref: 243 pid: 14261 timeout: 1153 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.954490:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726f:0x0].0x0 since=1s v=(1 0) 00010000:00010000:0.0:1785220581.954494:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x726f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.954498:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.954507:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.954527:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.954563:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+242:2407:x1871938519960704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 139us (280us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.954570:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100419 00010000:00010000:2.0:1785220581.954630:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.954639:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.954648:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.954655:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.954678:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.954829:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1215:3654:x1871938510704256:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9013us (9193us total) trans 4295070065 rc 0/0 00000100:00100000:2.0:1785220581.954837:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284486 00000004:00080000:2.0:1785220581.954883:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c3100/2, rc 0, transno 4295070065 00000100:00100000:2.0:1785220581.954892:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510704256:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.955006:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.955012:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.955020:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.955023:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.955032:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.955048:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.955054:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a51d0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x726f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.955070:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.955072:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.955074:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.955155:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220581.955167:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510706560:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220581.955200:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.956311:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510706560 00000100:00100000:2.0:1785220581.956319:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510706560 00000100:00100000:2.0:1785220581.956388:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284494 00000100:00100000:2.0:1785220581.956395:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1215:3655:x1871938510706560:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.956417:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29303 count 1 00010000:00010000:2.0:1785220581.956436:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.956451:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000044a1ff70/0xf62499ab9278e4f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e9bc expref: 241 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.956459:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7277:0x0].0x0 since=24s v=(0 0) 00010000:00010000:2.0:1785220581.956462:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7277:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.956466:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.956490:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000044a1ff70/0xf62499ab9278e4f0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e9bc expref: 241 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.956502:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000044a1ff70/0xf62499ab9278e4f0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e9bc expref: 241 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.956513:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510706688:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.956572:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e82bfd480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.956821:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510705152:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.956837:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.956927:0:8750:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 185 changes, 175 in progress, 0 in flight, 1091 done 00000100:00100000:3.0:1785220581.957298:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510705408:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.957311:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.957491:0:8750:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 185 changes, 16 in progress, 0 in flight, 159 done 00000100:00100000:0.0:1785220581.957600:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510705024:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.957612:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.957636:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510706944:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.957675:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.957740:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510707200:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.958068:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510705920:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.958080:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.958185:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510707456:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.958206:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.958263:0:14164:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510706944 00000100:00100000:1.0:1785220581.958269:0:14164:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510706944 00000100:00100000:1.0:1785220581.958305:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284495 00000100:00100000:1.0:1785220581.958311:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+21:3652:x1871938510706944:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.958329:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29344 count 1 00010000:00010000:1.0:1785220581.958347:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.958362:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006e639059/0xf62499ab9278fad0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f29f expref: 204 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.958375:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a0:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.958379:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.958384:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.958405:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006e639059/0xf62499ab9278fad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f29f expref: 204 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.958416:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006e639059/0xf62499ab9278fad0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f29f expref: 204 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.958427:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510707840:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.958468:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.958486:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510707072:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.958598:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510706816:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.958621:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.958639:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510707328:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.958656:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.958676:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510707456 00000100:00100000:0.0:1785220581.958681:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510707456 00000100:00100000:0.0:1785220581.958701:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510706816 00000100:00100000:0.0:1785220581.958704:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510706816 00000100:00100000:0.0:1785220581.958719:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510707328 00000100:00100000:0.0:1785220581.958721:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510707328 00000100:00100000:0.0:1785220581.958738:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284496 00000100:00100000:0.0:1785220581.958745:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+24:3652:x1871938510707456:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.958767:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29348 count 1 00010000:00010000:0.0:1785220581.958780:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.958792:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d72bf8f9/0xf62499ab9278fdb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d7 expref: 204 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.958799:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a4:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220581.958802:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.958807:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.958812:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284497 00010000:00010000:0.0:1785220581.958825:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d72bf8f9/0xf62499ab9278fdb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d7 expref: 204 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.958835:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d72bf8f9/0xf62499ab9278fdb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d7 expref: 204 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.958845:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0000_UUID:14162:1871938510707968:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.958869:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.958886:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284498 00000100:00100000:0.0:1785220581.958893:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+24:3655:x1871938510707328:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.958905:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29347 count 1 00010000:00010000:0.0:1785220581.958915:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.958927:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009d50eaec/0xf62499ab9278fd38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c9 expref: 204 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.958934:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a3:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220581.958937:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a3:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220581.958942:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+24:3655:x1871938510706816:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220581.958942:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.958957:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29343 count 1 00010000:00010000:0.0:1785220581.958960:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009d50eaec/0xf62499ab9278fd38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c9 expref: 204 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.958968:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009d50eaec/0xf62499ab9278fd38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c9 expref: 204 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.958970:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.958979:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510708096:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.958981:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f0825b96/0xf62499ab9278fa1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f291 expref: 204 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.958988:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x729f:0x0].0x0 since=21s v=(0 0) 00010000:00010000:3.0:1785220581.958991:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.958994:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.959000:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220581.959012:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f0825b96/0xf62499ab9278fa1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f291 expref: 204 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.959021:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f0825b96/0xf62499ab9278fa1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f291 expref: 204 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.959031:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510708224:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.959052:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f480 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.959065:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.959113:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510707200 00000100:00100000:2.0:1785220581.959118:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510707200 00000100:00100000:3.0:1785220581.959119:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510707712:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.959144:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284499 00000100:00100000:2.0:1785220581.959151:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+25:3654:x1871938510707200:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.959163:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29346 count 1 00010000:00010000:2.0:1785220581.959175:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.959193:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003dc76cc3/0xf62499ab9278fc58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2bb expref: 204 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.959200:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a2:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220581.959203:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.959207:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.959212:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220581.959226:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003dc76cc3/0xf62499ab9278fc58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2bb expref: 204 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.959235:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003dc76cc3/0xf62499ab9278fc58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2bb expref: 204 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.959236:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510707072 00000100:00100000:1.0:1785220581.959239:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510707072 00000100:00100000:2.0:1785220581.959245:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510708352:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.959261:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284500 00000100:00100000:1.0:1785220581.959266:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+26:3653:x1871938510707072:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.959267:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01240 going to sleep for 11 seconds 00002000:00080000:1.0:1785220581.959277:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29345 count 1 00010000:00010000:1.0:1785220581.959287:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.959298:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004a653046/0xf62499ab9278fbc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ad expref: 204 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.959305:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a1:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.959307:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.959311:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.959331:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004a653046/0xf62499ab9278fbc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ad expref: 204 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.959339:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004a653046/0xf62499ab9278fbc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ad expref: 204 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.959348:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510708480:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.959368:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdb40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.959385:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510707584:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.959408:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.959638:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510707584 00000100:00100000:0.0:1785220581.959643:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510707584 00000100:00100000:0.0:1785220581.959675:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284501 00000100:00100000:0.0:1785220581.959682:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+27:3653:x1871938510707584:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.959694:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29349 count 1 00010000:00010000:0.0:1785220581.959706:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.959717:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4448fdc/0xf62499ab9278fdfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2e5 expref: 204 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.959724:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a5:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220581.959726:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.959730:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.959748:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d4448fdc/0xf62499ab9278fdfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2e5 expref: 204 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.959774:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d4448fdc/0xf62499ab9278fdfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2e5 expref: 204 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.959784:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510708608:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.961220:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.962573:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.962886:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0000_UUID:14162:1871938510707968:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.962911:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.962951:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0001_UUID:14174:1871938510706688:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.962964:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510708096:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.962966:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.962975:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.963251:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510708352:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.963264:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.963927:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510708224:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.963944:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.965257:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510706432:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.965280:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.965327:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.965328:0:14161:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510707712 00000100:00100000:1.0:1785220581.965336:0:14161:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510707712 00000100:00100000:1.0:1785220581.965385:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284502 00000100:00100000:1.0:1785220581.965394:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+28:3654:x1871938510707712:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.965424:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29350 count 1 00010000:00010000:1.0:1785220581.965447:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.965464:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002e9f38fd/0xf62499ab9278fe8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2f3 expref: 204 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.965472:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a6:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.965476:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.965481:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.965512:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002e9f38fd/0xf62499ab9278fe8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2f3 expref: 204 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.965525:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002e9f38fd/0xf62499ab9278fe8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2f3 expref: 204 pid: 14177 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.965541:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510708736:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.965588:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0001_UUID:14192:1871938510706176:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.965603:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.965605:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bd780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.965630:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510708480:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.965643:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.965667:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510707840:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.965677:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.965793:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510706048:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.965805:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.965899:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960960 00000100:00100000:3.0:1785220581.965904:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960960 00010000:00010000:3.0:1785220581.965927:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004f9b1a14/0xf62499ab9278dd25 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e651 expref: 242 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.965962:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100420 00000100:00100000:3.0:1785220581.965970:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+242:2407:x1871938519960960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.965982:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.965987:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004f9b1a14/0xf62499ab9278dd25 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e651 expref: 242 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.965998:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004f9b1a14/0xf62499ab9278dd25 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e651 expref: 242 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.966020:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f9b1a14/0xf62499ab9278dd25 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e651 expref: 242 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.966034:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7271:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.966037:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7271:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.966043:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.966052:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.966082:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.966189:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+241:2407:x1871938519960960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 218us (860us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.966197:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100420 00010000:00010000:1.0:1785220581.966327:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.966341:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.966355:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.966363:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.966379:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.966400:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.966406:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.966414:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.966418:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.966426:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.966438:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.966444:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a51de lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7271:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.966461:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.966462:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.966465:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.966526:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510708608:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.966536:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1216:3652:x1871938510704896:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19175us (19404us total) trans 4295070066 rc 0/0 00010000:00010000:0.0:1785220581.966542:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.966544:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284488 00000004:00080000:2.0:1785220581.966772:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a85c00/2, rc 0, transno 4295070066 00000100:00100000:2.0:1785220581.966796:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510704896:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.966903:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510708864:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.966935:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.966984:0:14203:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510708864 00000100:00100000:1.0:1785220581.966990:0:14203:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510708864 00000100:00100000:1.0:1785220581.967028:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284503 00000100:00100000:1.0:1785220581.967036:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1216:3655:x1871938510708864:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.967053:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29304 count 1 00010000:00010000:1.0:1785220581.967070:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.967118:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f7cf813a/0xf62499ab9278e5c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea1e expref: 240 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.967127:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7278:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.967131:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7278:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.967136:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.967158:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f7cf813a/0xf62499ab9278e5c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea1e expref: 240 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.967169:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f7cf813a/0xf62499ab9278e5c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea1e expref: 240 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.967180:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510708992:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.967231:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8f8bdb40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.968397:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.968430:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.968461:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.968484:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519960832 00000100:00100000:2.0:1785220581.968489:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.968491:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519960832 00000100:00100000:2.0:1785220581.968508:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961088 00000100:00100000:2.0:1785220581.968511:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961088 00010000:00010000:3.0:1785220581.968518:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000071accfe9/0xf62499ab9278dc76 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5e1 expref: 242 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.968524:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000b91a239/0xf62499ab9278dea6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e6f9 expref: 242 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.968587:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961344 00000100:00100000:3.0:1785220581.968591:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961344 00010000:00010000:3.0:1785220581.968605:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000044a1ff70/0xf62499ab9278e4f0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e9bc expref: 243 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.969185:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961600 00000100:00100000:2.0:1785220581.969190:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961600 00010000:00010000:2.0:1785220581.969209:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d72bf8f9/0xf62499ab9278fdb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d7 expref: 205 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.969236:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100421 00000100:00100000:2.0:1785220581.969245:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+243:2406:x1871938519960832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.969257:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.969262:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000071accfe9/0xf62499ab9278dc76 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5e1 expref: 243 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969274:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000071accfe9/0xf62499ab9278dc76 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5e1 expref: 243 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969294:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000071accfe9/0xf62499ab9278dc76 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e5e1 expref: 243 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969311:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7270:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969315:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7270:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969320:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969329:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969354:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.969399:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+242:2406:x1871938519960832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 155us (1007us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.969406:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100421 00000100:00100000:2.0:1785220581.969462:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100422 00000100:00100000:2.0:1785220581.969469:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+241:2406:x1871938519961088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.969477:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.969481:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000b91a239/0xf62499ab9278dea6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e6f9 expref: 241 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969491:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000b91a239/0xf62499ab9278dea6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e6f9 expref: 241 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969510:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b91a239/0xf62499ab9278dea6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e6f9 expref: 241 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969520:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7272:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969523:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7272:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969527:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969534:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969554:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.969573:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+240:2406:x1871938519961088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (1142us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.969580:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100422 00000100:00100000:2.0:1785220581.969594:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100423 00000100:00100000:2.0:1785220581.969601:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+239:2406:x1871938519961344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.969609:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.969626:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000044a1ff70/0xf62499ab9278e4f0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e9bc expref: 239 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969637:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000044a1ff70/0xf62499ab9278e4f0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e9bc expref: 239 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969655:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000044a1ff70/0xf62499ab9278e4f0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e9bc expref: 239 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969665:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7277:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969668:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7277:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969672:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969680:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969698:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.969716:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+238:2406:x1871938519961344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (1254us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.969722:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100423 00000100:00100000:2.0:1785220581.969736:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100424 00000100:00100000:2.0:1785220581.969743:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+205:2406:x1871938519961600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.969751:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.969767:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d72bf8f9/0xf62499ab9278fdb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d7 expref: 205 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969777:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d72bf8f9/0xf62499ab9278fdb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d7 expref: 205 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969797:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d72bf8f9/0xf62499ab9278fdb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d7 expref: 205 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969807:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969810:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.969814:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969821:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.969847:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.969866:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+204:2406:x1871938519961600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 123us (1376us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.969873:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100424 00000100:00100000:2.0:1785220581.969915:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.969930:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961856 00000100:00100000:1.0:1785220581.969934:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961856 00010000:00010000:1.0:1785220581.969953:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f0825b96/0xf62499ab9278fa1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f291 expref: 204 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.969974:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100425 00000100:00100000:2.0:1785220581.969978:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.969981:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+204:2406:x1871938519961856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.969989:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.969994:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f0825b96/0xf62499ab9278fa1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f291 expref: 204 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.970003:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.970004:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f0825b96/0xf62499ab9278fa1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f291 expref: 204 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970023:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0825b96/0xf62499ab9278fa1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f291 expref: 204 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.970027:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.970035:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970038:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x729f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970042:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970049:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970067:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.970125:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.970161:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.970178:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519962112 00000100:00100000:2.0:1785220581.970182:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519962112 00010000:00010000:2.0:1785220581.970196:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004a653046/0xf62499ab9278fbc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ad expref: 204 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.970222:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519962496 00000100:00100000:2.0:1785220581.970225:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519962496 00010000:00010000:2.0:1785220581.970238:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cac16859/0xf62499ab9278e3fb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e95a expref: 238 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.970255:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961216 00000100:00100000:2.0:1785220581.970258:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961216 00010000:00010000:2.0:1785220581.970270:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000013aac52b/0xf62499ab9278e2ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8d5 expref: 239 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.970285:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961472 00000100:00100000:2.0:1785220581.970288:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961472 00010000:00010000:2.0:1785220581.970299:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006e639059/0xf62499ab9278fad0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f29f expref: 205 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.970315:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519962624 00000100:00100000:2.0:1785220581.970318:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519962624 00000100:00100000:1.0:1785220581.970323:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+206:2406:x1871938519961856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 342us (407us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.970329:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d4448fdc/0xf62499ab9278fdfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2e5 expref: 206 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.970330:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100425 00000100:00100000:2.0:1785220581.970341:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100426 00000100:00100000:2.0:1785220581.970346:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+205:2406:x1871938519962112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.970354:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.970359:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004a653046/0xf62499ab9278fbc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ad expref: 205 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.970373:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100427 00000100:00100000:1.0:1785220581.970380:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+239:2406:x1871938519962496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.970386:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.970391:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cac16859/0xf62499ab9278e3fb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e95a expref: 239 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970400:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cac16859/0xf62499ab9278e3fb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e95a expref: 239 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970417:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cac16859/0xf62499ab9278e3fb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e95a expref: 239 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970428:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7276:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970431:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7276:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970436:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970444:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970462:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.970487:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+238:2406:x1871938519962496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.970494:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100427 00000100:00100000:1.0:1785220581.970507:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100428 00000100:00100000:1.0:1785220581.970514:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9afc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+237:2407:x1871938519961216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.970521:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.970525:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000013aac52b/0xf62499ab9278e2ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8d5 expref: 237 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970535:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000013aac52b/0xf62499ab9278e2ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8d5 expref: 237 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970553:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000013aac52b/0xf62499ab9278e2ce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8d5 expref: 237 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970563:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7275:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970565:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7275:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970569:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970576:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970593:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.970609:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9afc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+236:2407:x1871938519961216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (581us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.970616:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100428 00000100:00100000:1.0:1785220581.970629:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100429 00000100:00100000:1.0:1785220581.970634:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+205:2407:x1871938519961472:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.970644:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.970648:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006e639059/0xf62499ab9278fad0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f29f expref: 205 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970657:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006e639059/0xf62499ab9278fad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f29f expref: 205 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970673:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e639059/0xf62499ab9278fad0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f29f expref: 205 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970682:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970685:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.970689:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970695:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.970712:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.970728:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+204:2407:x1871938519961472:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (604us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.970735:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100429 00000100:00100000:1.0:1785220581.970747:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100430 00000100:00100000:1.0:1785220581.970752:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+203:2406:x1871938519962624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.971245:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004a653046/0xf62499ab9278fbc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ad expref: 203 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.971260:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a653046/0xf62499ab9278fbc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ad expref: 203 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.971272:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.971276:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.971280:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.971287:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.971308:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.971334:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+202:2406:x1871938519962112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 987us (1355us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.971341:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100426 00000100:00100000:2.0:1785220581.971413:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.971789:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.971797:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d4448fdc/0xf62499ab9278fdfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2e5 expref: 201 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.971807:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d4448fdc/0xf62499ab9278fdfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2e5 expref: 201 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.971919:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4448fdc/0xf62499ab9278fdfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2e5 expref: 201 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.971931:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.971934:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.971939:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.971947:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.971976:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.971997:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+200:2406:x1871938519962624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1244us (1835us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.972001:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961728 00000100:00100000:1.0:1785220581.972004:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100430 00000100:00100000:2.0:1785220581.972006:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961728 00010000:00010000:2.0:1785220581.972025:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009d50eaec/0xf62499ab9278fd38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c9 expref: 200 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972028:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972037:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972046:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.972047:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100431 00000100:00100000:2.0:1785220581.972054:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+200:2407:x1871938519961728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.972055:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972062:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.972066:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009d50eaec/0xf62499ab9278fd38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c9 expref: 200 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972070:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.972075:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009d50eaec/0xf62499ab9278fd38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c9 expref: 200 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.972371:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1217:3653:x1871938510706304:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20227us (20354us total) trans 4295070067 rc 0/0 00000100:00100000:1.0:1785220581.972379:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284493 00010000:00010000:2.0:1785220581.972387:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d50eaec/0xf62499ab9278fd38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c9 expref: 200 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972397:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.972400:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.972402:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.972404:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972408:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972412:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972416:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.972420:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972429:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972442:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972448:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5201 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7276:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972465:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.972467:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.972469:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.972479:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972488:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972494:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.972496:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972503:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.972514:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2407:x1871938519961728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 460us (1101us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220581.972516:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220581.972521:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100431 00000100:00100000:2.0:1785220581.972558:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.972573:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519961984 00000100:00100000:2.0:1785220581.972576:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519961984 00010000:00010000:2.0:1785220581.972590:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003dc76cc3/0xf62499ab9278fc58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2bb expref: 199 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.972603:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+29:3653:x1871938510707072:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13336us (13390us total) trans 4295070650 rc 0/0 00000100:00100000:2.0:1785220581.972610:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100432 00000100:00100000:1.0:1785220581.972610:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284500 00000100:00100000:2.0:1785220581.972685:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2407:x1871938519961984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.972695:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.972699:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003dc76cc3/0xf62499ab9278fc58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2bb expref: 199 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972705:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972710:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003dc76cc3/0xf62499ab9278fc58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2bb expref: 199 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972715:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972723:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dc76cc3/0xf62499ab9278fc58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2bb expref: 199 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972723:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972732:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972733:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.972736:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.972741:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972748:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.972751:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.972780:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.972798:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+198:2407:x1871938519961984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.972808:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100432 00010000:00010000:2.0:1785220581.972827:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.972836:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.972966:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+29:3652:x1871938510706944:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14654us (15289us total) trans 4295070651 rc 0/0 00000100:00100000:1.0:1785220581.972973:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284495 00010000:00010000:1.0:1785220581.973002:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973005:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973011:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973013:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973019:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973026:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220581.973026:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973037:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:1.0:1785220581.973044:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220581.973048:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973054:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.973058:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973070:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973081:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.973231:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+29:3654:x1871938510707200:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14080us (14164us total) trans 4295070652 rc 0/0 00000100:00100000:2.0:1785220581.973238:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284499 00010000:00010000:2.0:1785220581.973255:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220581.973260:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973268:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.973271:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973280:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973281:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091eec52a/0xf62499ab927a5232 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973292:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973297:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.973300:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee522aef/0xf62499ab927a520f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973300:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973307:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.973311:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973313:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.973316:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973319:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973323:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.973325:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973329:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973332:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973336:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a522b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973341:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973349:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a5224 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973353:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.973354:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.973356:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.973361:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.973363:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.973365:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.973398:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601ce00/2, rc 0, transno 4295070651 00000100:00100000:2.0:1785220581.973410:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510706944:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.973506:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+29:3655:x1871938510706816:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14686us (14883us total) trans 4295070653 rc 0/0 00000100:00100000:1.0:1785220581.973512:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284497 00000100:00100000:2.0:1785220581.973532:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:1.0:1785220581.973544:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601e300/2, rc 0, transno 4295070650 00000100:00100000:1.0:1785220581.973549:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510707072:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.973559:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519962240 00000100:00100000:1.0:1785220581.973562:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519962240 00000100:00100000:2.0:1785220581.973573:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220581.973578:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000980ff11b/0xf62499ab9278dfe8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e793 expref: 236 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.973588:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519962368 00000100:00100000:2.0:1785220581.973591:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519962368 00000100:00100000:1.0:1785220581.973597:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100433 00000100:00100000:1.0:1785220581.973603:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+237:2407:x1871938519962240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.973604:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005efb3a7d/0xf62499ab9278e1d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e87a expref: 237 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973612:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.973616:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000980ff11b/0xf62499ab9278dfe8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e793 expref: 237 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973625:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000980ff11b/0xf62499ab9278dfe8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e793 expref: 237 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973638:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000980ff11b/0xf62499ab9278dfe8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e793 expref: 237 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973653:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7273:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.973656:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7273:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.973661:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973669:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.973688:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.973691:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100434 00000100:00100000:2.0:1785220581.973698:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+236:2407:x1871938519962368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220581.973705:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220581.973706:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+236:2407:x1871938519962240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (174us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220581.973710:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005efb3a7d/0xf62499ab9278e1d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e87a expref: 236 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.973711:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100433 00010000:00010000:2.0:1785220581.973720:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005efb3a7d/0xf62499ab9278e1d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e87a expref: 235 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973733:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005efb3a7d/0xf62499ab9278e1d2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e87a expref: 235 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220581.973738:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601f480/2, rc 0, transno 4295070067 00010000:00010000:2.0:1785220581.973743:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7274:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.973746:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7274:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220581.973746:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510706304:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.973751:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.973808:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.973820:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510709120:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.973827:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.973898:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220581.973918:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+234:2407:x1871938519962368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 220us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.973925:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100434 00000004:00080000:2.0:1785220581.973960:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601d880/2, rc 0, transno 4295070653 00000100:00100000:2.0:1785220581.973964:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510706816:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220581.973968:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.973992:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510709120 00000100:00100000:1.0:1785220581.973996:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510709120 00000100:00100000:1.0:1785220581.974021:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284504 00000100:00100000:1.0:1785220581.974027:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1217:3653:x1871938510709120:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.974047:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29305 count 1 00010000:00010000:1.0:1785220581.974068:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.974346:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fbffe78f/0xf62499ab9278e6b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea95 expref: 233 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.974354:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7279:0x0].0x0 since=24s v=(0 0) 00010000:00010000:1.0:1785220581.974357:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7279:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.974360:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.974386:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fbffe78f/0xf62499ab9278e6b0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea95 expref: 233 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.974398:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fbffe78f/0xf62499ab9278e6b0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea95 expref: 233 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.974410:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510709248:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.974447:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.974540:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.974549:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.974558:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.974565:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.974577:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.975115:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.975122:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.975229:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.975239:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.975248:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.975255:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975351:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.975356:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975370:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975386:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975393:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5239 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975411:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.975414:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975421:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.975423:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975431:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975442:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.975450:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.975451:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.975512:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.975591:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.975736:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+29:3653:x1871938510707584:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16053us (16325us total) trans 4295070654 rc 0/0 00000100:00100000:0.0:1785220581.975745:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284501 00000100:00100000:2.0:1785220581.976076:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.976174:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.976186:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976196:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976203:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976209:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976225:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.976345:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1218:3652:x1871938510705792:12345-0@lo:6:osp-syn-1-0.0 Request processed in 26488us (27011us total) trans 4295070068 rc 0/0 00000100:00100000:0.0:1785220581.976352:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284490 00010000:00010000:0.0:1785220581.976375:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976384:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976391:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976399:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.976412:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220581.976603:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220581.976609:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977033:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.977038:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977046:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977061:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977072:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a51ec lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7275:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977148:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.977152:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977160:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.977163:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977171:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977182:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977189:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c8c95977/0xf62499ab927a51fa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7273:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.977202:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.977203:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.977205:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.977239:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a84700/2, rc 0, transno 4295070068 00000100:00100000:2.0:1785220581.977252:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510705792:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.977268:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.977289:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510708992:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.977308:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.977310:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220581.977320:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510709376:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.977331:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510708736:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.977344:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.977355:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.977362:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519962752 00000100:00100000:1.0:1785220581.977367:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519962752 00010000:00010000:1.0:1785220581.977391:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002e9f38fd/0xf62499ab9278fe8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2f3 expref: 198 pid: 14177 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.977418:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100435 00000100:00100000:1.0:1785220581.977425:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e10850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+198:2407:x1871938519962752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.977437:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.977442:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002e9f38fd/0xf62499ab9278fe8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2f3 expref: 198 pid: 14177 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977452:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002e9f38fd/0xf62499ab9278fe8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2f3 expref: 198 pid: 14177 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977469:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e9f38fd/0xf62499ab9278fe8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2f3 expref: 198 pid: 14177 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977487:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.977492:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.977497:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977505:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977525:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.977557:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e10850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2407:x1871938519962752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 133us (1482us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.977564:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100435 00000100:00100000:0.0:1785220581.977586:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+1218:3655:x1871938510705536:12345-0@lo:6:osp-syn-1-0.0 Request processed in 26367us (28107us total) trans 4295070069 rc 0/0 00000100:00100000:0.0:1785220581.977592:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284492 00000100:00100000:1.0:1785220581.977603:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510709376 00000100:00100000:1.0:1785220581.977606:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510709376 00000100:00100000:1.0:1785220581.977628:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284505 00000100:00100000:1.0:1785220581.977634:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1218:3652:x1871938510709376:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.977652:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29306 count 1 00010000:00010000:1.0:1785220581.977669:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977683:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f3faa75e/0xf62499ab9278e774 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb05 expref: 233 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.977683:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977691:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x727a:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.977693:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977695:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.977700:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.977707:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.977714:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977719:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f3faa75e/0xf62499ab9278e774 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb05 expref: 233 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.977726:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.977730:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f3faa75e/0xf62499ab9278e774 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb05 expref: 233 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.977741:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510709504:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.977791:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba0c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.977807:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.977815:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.977923:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220581.977933:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220581.977939:0:14204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a51e5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7272:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.978035:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1219:3652:x1871938510705280:12345-0@lo:6:osp-syn-1-0.0 Request processed in 30378us (30473us total) trans 4295070070 rc 0/0 00000100:00100000:3.0:1785220581.978042:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284489 00010000:00010000:3.0:1785220581.978066:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978074:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978082:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978118:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978135:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220581.978207:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1219:3655:x1871938510705664:12345-0@lo:6:osp-syn-1-0.0 Request processed in 27772us (28774us total) trans 4295070071 rc 0/0 00000100:00100000:3.0:1785220581.978216:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284491 00010000:00010000:3.0:1785220581.978236:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978244:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978252:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978259:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.978273:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220581.978339:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+1219:3655:x1871938510706560:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21944us (23138us total) trans 4295070072 rc 0/0 00000100:00100000:3.0:1785220581.978345:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284494 00000004:00080000:3.0:1785220581.978381:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601d500/2, rc 0, transno 4295070652 00000100:00100000:3.0:1785220581.978386:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510707200:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.978991:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+29:3655:x1871938510707328:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20097us (20332us total) trans 4295070655 rc 0/0 00000100:00100000:0.0:1785220581.978998:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284498 00010000:00010000:0.0:1785220581.979017:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220581.979023:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.979031:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.979034:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.979043:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.979059:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.979067:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062842ed7/0xf62499ab927a521d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979277:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979287:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979298:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220581.979384:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+29:3654:x1871938510707712:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13991us (16816us total) trans 4295070656 rc 0/0 00000100:00100000:1.0:1785220581.979395:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284502 00010000:00010000:1.0:1785220581.979414:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220581.979419:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979426:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.979429:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979437:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979450:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979457:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a51f3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7274:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979472:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.979476:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979482:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.979486:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979498:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979507:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979514:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5208 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7277:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979524:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.979527:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979533:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.979535:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979542:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979552:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979559:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6fec17/0xf62499ab927a5240 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979577:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.979579:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.979580:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.979610:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510709248:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.979629:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.979650:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519962880 00000100:00100000:1.0:1785220581.979656:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519962880 00010000:00010000:1.0:1785220581.979682:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f7cf813a/0xf62499ab9278e5c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea1e expref: 234 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.979709:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100436 00000100:00100000:1.0:1785220581.979717:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+234:2407:x1871938519962880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.979729:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.979735:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f7cf813a/0xf62499ab9278e5c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea1e expref: 234 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.979746:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f7cf813a/0xf62499ab9278e5c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea1e expref: 234 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980040:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f7cf813a/0xf62499ab9278e5c2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea1e expref: 234 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980059:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7278:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.980064:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7278:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.980069:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980079:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980132:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220581.980165:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+233:2407:x1871938519962880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 448us (2337us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.980172:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100436 00010000:00010000:1.0:1785220581.980192:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980200:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980209:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980216:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.980228:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220581.980249:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.980256:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980263:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.980266:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980273:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980284:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980291:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2d0c599/0xf62499ab927a5247 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7278:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980304:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.980306:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.980308:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.980321:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1219:3655:x1871938510708864:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13287us (13386us total) trans 4295070073 rc 0/0 00000100:00100000:1.0:1785220581.980328:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284503 00000004:00080000:2.0:1785220581.980332:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4b480/2, rc 0, transno 4295070070 00000100:00100000:2.0:1785220581.980342:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510705280:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.980344:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220581.980370:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601e680/2, rc 0, transno 4295070654 00000100:00100000:1.0:1785220581.980375:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510707584:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.980421:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220581.980518:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.980520:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.980522:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.980540:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510709632:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.980540:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.980548:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.980556:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.980563:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.980573:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.980581:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1785220581.980611:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601f800/2, rc 0, transno 4295070656 00000100:00100000:3.0:1785220581.980615:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510707712:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.980643:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be5500/2, rc 0, transno 4295070073 00000100:00100000:3.0:1785220581.980650:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510708864:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.980661:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601c380/2, rc 0, transno 4295070655 00000100:00100000:3.0:1785220581.980665:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510707328:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220581.980674:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601ea00/2, rc 0, transno 4295070072 00000100:00100000:3.0:1785220581.980677:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510706560:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.980687:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a86d80/2, rc 0, transno 4295070069 00000100:00100000:3.0:1785220581.980690:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510705536:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220581.980698:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a86300/2, rc 0, transno 4295070071 00000100:00100000:3.0:1785220581.980701:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510705664:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.980702:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.980707:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980713:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.980717:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980724:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980736:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.980744:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5216 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.980846:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220581.981009:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510709504:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.981027:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.981046:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963008 00000100:00100000:1.0:1785220581.981050:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963008 00010000:00010000:1.0:1785220581.981068:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fbffe78f/0xf62499ab9278e6b0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea95 expref: 233 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.981245:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963136 00000100:00100000:1.0:1785220581.981249:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963136 00010000:00010000:1.0:1785220581.981264:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f3faa75e/0xf62499ab9278e774 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb05 expref: 234 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.981297:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100437 00000100:00100000:3.0:1785220581.981308:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+234:2407:x1871938519963008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.981318:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.981323:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fbffe78f/0xf62499ab9278e6b0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea95 expref: 234 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981333:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fbffe78f/0xf62499ab9278e6b0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea95 expref: 234 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981348:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fbffe78f/0xf62499ab9278e6b0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ea95 expref: 234 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.981354:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+29:3652:x1871938510707456:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22609us (23146us total) trans 4295070657 rc 0/0 00010000:00010000:3.0:1785220581.981358:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7279:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220581.981361:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284496 00010000:00010000:3.0:1785220581.981362:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7279:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.981367:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.981373:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510709632 00010000:00010000:3.0:1785220581.981375:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.981376:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510709632 00010000:00010000:3.0:1785220581.981392:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.981396:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284506 00000100:00100000:0.0:1785220581.981401:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1219:3654:x1871938510709632:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.981413:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+233:2407:x1871938519963008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (993us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.981420:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100437 00002000:00080000:0.0:1785220581.981420:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29307 count 1 00000100:00100000:3.0:1785220581.981433:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100438 00000100:00100000:3.0:1785220581.981439:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+232:2407:x1871938519963136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.981448:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.981453:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f3faa75e/0xf62499ab9278e774 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb05 expref: 232 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981462:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f3faa75e/0xf62499ab9278e774 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb05 expref: 232 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981474:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f3faa75e/0xf62499ab9278e774 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb05 expref: 232 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981483:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.981487:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.981491:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981503:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981520:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.981520:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.981522:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.981524:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.981537:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+231:2407:x1871938519963136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (692us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.981543:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100438 00000100:00100000:3.0:1785220581.981573:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510709760:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.981602:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510709888:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.981604:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.981629:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1785220581.981635:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510710144:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.981656:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.981675:0:14267:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510709760 00000100:00100000:3.0:1785220581.981679:0:14267:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510709760 00010000:00010000:0.0:1785220581.981698:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.981700:0:14267:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510710144 00000100:00100000:3.0:1785220581.981703:0:14267:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510710144 00010000:00010000:0.0:1785220581.981713:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002bc5cdee/0xf62499ab9278e8bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb7c expref: 230 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.981718:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284507 00010000:00010000:0.0:1785220581.981721:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x727b:0x0].0x0 since=23s v=(0 0) 00000100:00100000:3.0:1785220581.981723:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90e5ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1221:3655:x1871938510709760:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.981724:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.981728:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.981734:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29308 count 1 00010000:00010000:3.0:1785220581.981745:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.981749:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002bc5cdee/0xf62499ab9278e8bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb7c expref: 230 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981767:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e748b208/0xf62499ab9278e9a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebe5 expref: 230 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981779:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x727c:0x0].0x0 since=23s v=(0 0) 00010000:00010000:3.0:1785220581.981782:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.981785:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981803:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e748b208/0xf62499ab9278e9a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebe5 expref: 230 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.981815:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e748b208/0xf62499ab9278e9a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebe5 expref: 230 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.981890:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510710400:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.981926:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.981940:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284508 00000100:00100000:2.0:1785220581.981948:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1221:3655:x1871938510710144:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.981960:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29311 count 1 00010000:00010000:2.0:1785220581.981972:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.981989:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008d0cfc14/0xf62499ab9278ec75 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed4a expref: 230 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.981996:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x727f:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220581.981999:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.982002:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.982027:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002bc5cdee/0xf62499ab9278e8bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb7c expref: 230 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.982038:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510710272:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.982054:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.982063:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.982063:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d260c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220581.982072:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.982080:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.982080:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.982249:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008d0cfc14/0xf62499ab9278ec75 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed4a expref: 230 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.982260:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008d0cfc14/0xf62499ab9278ec75 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed4a expref: 230 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.982270:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510710528:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.982307:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.982333:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.982361:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601d180/2, rc 0, transno 4295070657 00000100:00100000:2.0:1785220581.982367:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510707456:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.982375:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220581.982378:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510709888 00000100:00100000:2.0:1785220581.982380:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510710016:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.982381:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510709888 00000100:00100000:2.0:1785220581.982402:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220581.982499:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.982508:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.982521:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.982536:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.982548:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220581.982637:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1224:3653:x1871938510709120:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8609us (8667us total) trans 4295070075 rc 0/0 00000100:00100000:1.0:1785220581.982644:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284504 00000100:00100000:0.0:1785220581.982713:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1223:3654:x1871938510704384:12345-0@lo:6:osp-syn-1-0.0 Request processed in 36667us (37014us total) trans 4295070074 rc 0/0 00000100:00100000:0.0:1785220581.982719:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284487 00000100:00100000:3.0:1785220581.982806:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510710016 00000100:00100000:3.0:1785220581.982810:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510710016 00000100:00100000:2.0:1785220581.982905:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284509 00000100:00100000:2.0:1785220581.982914:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1223:3652:x1871938510709888:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.982928:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29309 count 1 00010000:00010000:2.0:1785220581.982943:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.982944:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284510 00000100:00100000:3.0:1785220581.982951:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1223:3652:x1871938510710016:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220581.982955:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006b074804/0xf62499ab9278ea45 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec40 expref: 230 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220581.982960:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29310 count 1 00010000:00010000:2.0:1785220581.982963:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x727d:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220581.982965:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.982969:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.982970:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.982981:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd5cdb08/0xf62499ab9278ebb8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ecd3 expref: 230 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.982988:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006b074804/0xf62499ab9278ea45 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec40 expref: 230 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.982993:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x727e:0x0].0x0 since=23s v=(0 0) 00010000:00010000:3.0:1785220581.982996:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.982998:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006b074804/0xf62499ab9278ea45 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec40 expref: 230 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.983000:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.983009:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510710656:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.983016:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd5cdb08/0xf62499ab9278ebb8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ecd3 expref: 230 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.983025:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd5cdb08/0xf62499ab9278ebb8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ecd3 expref: 230 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.983031:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220581.983034:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510710784:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220581.983052:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00000004:00080000:3.0:1785220581.983078:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c3480/2, rc 0, transno 4295070074 00000100:00100000:3.0:1785220581.983107:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510704384:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220581.983594:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220581.983601:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983610:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.983615:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983624:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983641:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983648:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a51d7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7270:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983666:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.983671:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983677:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.983680:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983688:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983698:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983706:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008e0e7058/0xf62499ab927a524e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7279:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983732:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.983734:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.983735:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.983745:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983762:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983771:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983781:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.983803:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.983923:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1224:3652:x1871938510709376:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6288us (6565us total) trans 4295070076 rc 0/0 00000100:00100000:1.0:1785220581.983930:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284505 00000004:00080000:1.0:1785220581.983967:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be5f80/2, rc 0, transno 4295070075 00000100:00100000:1.0:1785220581.983973:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510709120:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.984311:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510711040:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.984343:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220581.984368:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7f800/2, rc 0, transno 4295070076 00000100:00100000:2.0:1785220581.984377:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510709376:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.984410:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220581.984417:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510711168:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.984438:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.984460:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.984467:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510710912:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.984489:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.984582:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510711040 00000100:00100000:1.0:1785220581.984588:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510711040 00000100:00100000:0.0:1785220581.984600:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510711168 00000100:00100000:0.0:1785220581.984603:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510711168 00000100:00100000:1.0:1785220581.984623:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510710912 00000100:00100000:1.0:1785220581.984627:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510710912 00000100:00100000:0.0:1785220581.984634:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284511 00000100:00100000:0.0:1785220581.984641:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1226:3652:x1871938510711040:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.984659:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29313 count 1 00010000:00010000:0.0:1785220581.984677:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.984680:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220581.984686:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.984693:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.984693:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007865692b/0xf62499ab9278ee43 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee15 expref: 230 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.984697:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.984701:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7281:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.984704:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7281:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.984705:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.984709:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.984718:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.984725:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5255 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.984731:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007865692b/0xf62499ab9278ee43 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee15 expref: 230 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.984745:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007865692b/0xf62499ab9278ee43 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee15 expref: 230 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.984747:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.984749:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220581.984750:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.984770:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510711296:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.984827:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.984848:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.984920:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.985210:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e06a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510711808:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.985299:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.985318:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510712320:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.985337:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.985354:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1785220581.985360:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510711680:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.985379:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.985394:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510712192:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220581.985414:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220581.985434:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510711808 00000100:00100000:0.0:1785220581.985437:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510711808 00000100:00100000:1.0:1785220581.985521:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510712320 00000100:00100000:1.0:1785220581.985526:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510712320 00000100:00100000:1.0:1785220581.985544:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510711680 00000100:00100000:1.0:1785220581.985547:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510711680 00000100:00100000:1.0:1785220581.985561:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510712192 00000100:00100000:1.0:1785220581.985563:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510712192 00000100:00100000:1.0:1785220581.985579:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284512 00000100:00100000:1.0:1785220581.985586:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1226:3652:x1871938510711168:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.985602:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29314 count 1 00010000:00010000:1.0:1785220581.985616:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.985630:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000025f72b10/0xf62499ab9278eec1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee69 expref: 230 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.985643:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7282:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.985647:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7282:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.985651:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.985674:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000025f72b10/0xf62499ab9278eec1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee69 expref: 230 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.985685:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000025f72b10/0xf62499ab9278eec1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee69 expref: 230 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.985696:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510712448:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.985736:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.986112:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284513 00000100:00100000:1.0:1785220581.986121:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1226:3654:x1871938510710912:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220581.986145:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29312 count 1 00010000:00010000:1.0:1785220581.986161:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986173:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d07e59f7/0xf62499ab9278ed4e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edba expref: 230 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986180:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7280:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220581.986184:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7280:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.986188:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986288:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d07e59f7/0xf62499ab9278ed4e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edba expref: 230 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986303:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d07e59f7/0xf62499ab9278ed4e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edba expref: 230 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.986315:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510712576:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.986347:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.986367:0:14223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284514 00000100:00100000:1.0:1785220581.986378:0:14223:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb67800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+32:3653:x1871938510712320:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.986390:0:14223:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29358 count 1 00010000:00010000:1.0:1785220581.986411:0:14223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986424:0:14223:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006fa01f0c/0xf62499ab9279035f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f363 expref: 196 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986436:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ae:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.986439:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.986443:0:14223:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986465:0:14223:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006fa01f0c/0xf62499ab9279035f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f363 expref: 196 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986478:0:14223:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006fa01f0c/0xf62499ab9279035f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f363 expref: 196 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.986493:0:14223:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0000_UUID:14223:1871938510712704:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.986517:0:14223:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bad80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.986539:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284515 00000100:00100000:1.0:1785220581.986547:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+32:3653:x1871938510711680:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.986564:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29353 count 1 00010000:00010000:1.0:1785220581.986575:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986592:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009be89994/0xf62499ab92790095 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f31d expref: 196 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986600:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a9:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.986602:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.986606:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986626:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009be89994/0xf62499ab92790095 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f31d expref: 196 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986638:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009be89994/0xf62499ab92790095 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f31d expref: 196 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.986652:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0000_UUID:14248:1871938510712832:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.986674:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bacc0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.986691:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284516 00000100:00100000:1.0:1785220581.986697:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+32:3653:x1871938510712192:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.986716:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29357 count 1 00010000:00010000:1.0:1785220581.986728:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986745:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2f47ec9/0xf62499ab927902e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f355 expref: 196 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986752:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ad:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.986764:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.986768:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986792:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f2f47ec9/0xf62499ab927902e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f355 expref: 196 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.986801:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f2f47ec9/0xf62499ab927902e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f355 expref: 196 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.986811:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510712960:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.986835:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.990024:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510711424:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.990084:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.990139:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510711936:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.990162:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.990185:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1785220581.990191:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e04700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510711552:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.990212:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.990228:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510712064:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.990247:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.990293:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220581.990364:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510710400:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.990390:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.990419:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963264 00000100:00100000:2.0:1785220581.990425:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963264 00000100:00100000:3.0:1785220581.990447:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220581.990456:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002bc5cdee/0xf62499ab9278e8bd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb7c expref: 231 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.990463:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.990482:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963392 00000100:00100000:3.0:1785220581.990484:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963520 00000100:00100000:2.0:1785220581.990485:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963392 00000100:00100000:3.0:1785220581.990487:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963520 00010000:00010000:2.0:1785220581.990496:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e748b208/0xf62499ab9278e9a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebe5 expref: 233 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990499:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008d0cfc14/0xf62499ab9278ec75 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed4a expref: 233 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.990516:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100439 00000100:00100000:3.0:1785220581.990521:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100440 00000100:00100000:2.0:1785220581.990524:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+233:2406:x1871938519963264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220581.990526:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+233:2406:x1871938519963392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.990538:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220581.990538:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.990544:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e748b208/0xf62499ab9278e9a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebe5 expref: 233 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.990544:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002bc5cdee/0xf62499ab9278e8bd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb7c expref: 233 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990556:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e748b208/0xf62499ab9278e9a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebe5 expref: 233 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.990556:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002bc5cdee/0xf62499ab9278e8bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb7c expref: 233 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990578:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e748b208/0xf62499ab9278e9a4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ebe5 expref: 233 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.990580:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.990585:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.990590:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.990591:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990593:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.990597:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.990601:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990604:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990620:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220581.990628:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220581.990658:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+232:2406:x1871938519963392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 130us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.990665:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100440 00000100:00100000:3.0:1785220581.990682:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100441 00000100:00100000:3.0:1785220581.990688:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+231:2407:x1871938519963520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220581.990696:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220581.990700:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008d0cfc14/0xf62499ab9278ec75 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed4a expref: 231 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990714:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008d0cfc14/0xf62499ab9278ec75 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed4a expref: 231 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.990726:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.990729:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.990733:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.990736:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510711424 00010000:00010000:3.0:1785220581.990740:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.990740:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510711424 00010000:00010000:3.0:1785220581.990750:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.990948:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510711936 00000100:00100000:0.0:1785220581.990953:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510711936 00000100:00100000:0.0:1785220581.990971:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510711552 00000100:00100000:0.0:1785220581.990973:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510711552 00000100:00100000:0.0:1785220581.990987:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510712064 00000100:00100000:0.0:1785220581.990990:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510712064 00000100:00100000:0.0:1785220581.991006:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284517 00000100:00100000:0.0:1785220581.991013:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+36:3653:x1871938510711808:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220581.991036:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29354 count 1 00010000:00010000:0.0:1785220581.991060:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.991072:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb4b0d23/0xf62499ab92790128 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f32b expref: 196 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.991079:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72aa:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220581.991082:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.991124:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.991155:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cb4b0d23/0xf62499ab92790128 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f32b expref: 196 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.991168:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cb4b0d23/0xf62499ab92790128 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f32b expref: 196 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.991181:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510713088:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.991242:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284518 00000100:00100000:1.0:1785220581.991252:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+36:3654:x1871938510711424:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.991254:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+231:2406:x1871938519963264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 731us (963us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220581.991261:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100439 00002000:00080000:1.0:1785220581.991264:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29351 count 1 00010000:00010000:1.0:1785220581.991276:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991287:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c3d46ee1/0xf62499ab9278ff30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f301 expref: 196 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991292:0:14206:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d0cfc14/0xf62499ab9278ec75 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed4a expref: 230 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991294:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a7:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.991297:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.991301:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.991304:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510710528:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.991316:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991321:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c3d46ee1/0xf62499ab9278ff30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f301 expref: 196 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991331:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991331:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c3d46ee1/0xf62499ab9278ff30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f301 expref: 196 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991340:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.991342:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510713216:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.991349:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991357:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991373:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220581.991377:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.991392:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284519 00000100:00100000:1.0:1785220581.991399:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+36:3654:x1871938510711936:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220581.991413:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29355 count 1 00010000:00010000:1.0:1785220581.991423:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991433:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004e3ea216/0xf62499ab927901b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f339 expref: 196 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991438:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ab:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220581.991441:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.991444:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991459:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004e3ea216/0xf62499ab927901b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f339 expref: 196 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991468:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004e3ea216/0xf62499ab927901b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f339 expref: 196 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.991476:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510713344:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220581.991497:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220581.991514:0:14193:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284520 00000100:00100000:1.0:1785220581.991521:0:14193:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+36:3654:x1871938510711552:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220581.991522:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+1227:3655:x1871938510710144:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9576us (9865us total) trans 4295070077 rc 0/0 00000100:00100000:2.0:1785220581.991528:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284508 00002000:00080000:1.0:1785220581.991532:0:14193:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29352 count 1 00010000:00010000:1.0:1785220581.991543:0:14193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.991544:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284521 00000100:00100000:2.0:1785220581.991550:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+36:3654:x1871938510712064:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220581.991559:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29356 count 1 00010000:00010000:1.0:1785220581.991560:0:14193:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000df1d902f/0xf62499ab92790017 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f30f expref: 196 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991566:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72a8:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220581.991568:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991569:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.991573:0:14193:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991579:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000be6cf53b/0xf62499ab92790278 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f347 expref: 196 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991589:0:14193:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000df1d902f/0xf62499ab92790017 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f30f expref: 196 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991590:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ac:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220581.991593:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.991597:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.991598:0:14193:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000df1d902f/0xf62499ab92790017 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f30f expref: 196 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.991608:0:14193:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510713472:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.991624:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000be6cf53b/0xf62499ab92790278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f347 expref: 196 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.991631:0:14193:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220581.991635:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000be6cf53b/0xf62499ab92790278 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f347 expref: 196 pid: 14204 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.991644:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510713600:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.991665:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.991687:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510710656:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.991700:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991727:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.991733:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991741:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220581.991745:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991926:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991942:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991948:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a526a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.991970:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.991971:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.991973:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.992004:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7ca80/2, rc 0, transno 4295070077 00000100:00100000:2.0:1785220581.992015:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510710144:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.992061:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510713728:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.992107:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.992232:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+229:2407:x1871938519963520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1543us (1766us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220581.992241:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100441 00010000:00010000:3.0:1785220581.992265:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992273:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992280:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992287:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992309:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.992401:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90e5ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1227:3655:x1871938510709760:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10677us (10796us total) trans 4295070078 rc 0/0 00000100:00100000:3.0:1785220581.992408:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284507 00000100:00100000:3.0:1785220581.992420:0:14267:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510713728 00000100:00100000:3.0:1785220581.992425:0:14267:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510713728 00000100:00100000:3.0:1785220581.992452:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284522 00000100:00100000:3.0:1785220581.992458:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1227:3654:x1871938510713728:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220581.992470:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29315 count 1 00010000:00010000:3.0:1785220581.992484:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992496:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c1396b89/0xf62499ab9278ef62 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eea8 expref: 228 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.992497:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220581.992502:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992503:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7283:0x0].0x0 since=23s v=(0 0) 00010000:00010000:3.0:1785220581.992506:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7283:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.992510:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220581.992511:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.992514:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.992521:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992531:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c1396b89/0xf62499ab9278ef62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eea8 expref: 228 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.992533:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.992540:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a5263 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.992542:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c1396b89/0xf62499ab9278ef62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eea8 expref: 228 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.992557:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220581.992559:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510713856:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.992559:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220581.992561:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220581.992578:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7fb80/2, rc 0, transno 4295070078 00000100:00100000:2.0:1785220581.992587:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510709760:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.992589:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220581.992632:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220581.992640:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510713984:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.992641:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510710784:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220581.992658:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220581.992665:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220581.992752:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220581.992850:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510711296:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220581.992865:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.993124:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220581.993140:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510713984 00000100:00100000:0.0:1785220581.993144:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510713984 00000100:00100000:0.0:1785220581.993173:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284523 00000100:00100000:0.0:1785220581.993180:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1228:3654:x1871938510713984:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.993194:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29316 count 1 00010000:00010000:0.0:1785220581.993206:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993219:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000903ea35a/0xf62499ab9278f011 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eef5 expref: 228 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993225:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7284:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.993228:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7284:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.993231:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993252:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000903ea35a/0xf62499ab9278f011 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eef5 expref: 228 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993261:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000903ea35a/0xf62499ab9278f011 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eef5 expref: 228 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.993272:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510714112:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.993297:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220581.993398:0:14162:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002bc5cdee/0xf62499ab9278e8bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eb7c expref: 228 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.993409:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510710272:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220581.993422:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993430:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993436:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993445:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993452:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993467:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220581.993571:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1229:3654:x1871938510709632:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12169us (12996us total) trans 4295070079 rc 0/0 00000100:00100000:0.0:1785220581.993578:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284506 00000100:00100000:3.0:1785220581.993629:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:2.0:1785220581.993736:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7df80/2, rc 0, transno 4295070079 00000100:00100000:2.0:1785220581.993746:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510709632:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220581.993946:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220581.993952:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993961:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220581.993965:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993974:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993988:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.993996:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a525c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994013:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.994014:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220581.994016:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.994033:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963776 00000100:00100000:0.0:1785220581.994038:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963776 00010000:00010000:0.0:1785220581.994058:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007865692b/0xf62499ab9278ee43 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee15 expref: 228 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.994070:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963648 00000100:00100000:1.0:1785220581.994074:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963648 00000100:00100000:0.0:1785220581.994107:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100442 00000100:00100000:0.0:1785220581.994116:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac547450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+229:2407:x1871938519963776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220581.994119:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd5cdb08/0xf62499ab9278ebb8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ecd3 expref: 229 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994125:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220581.994130:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007865692b/0xf62499ab9278ee43 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee15 expref: 229 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994141:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007865692b/0xf62499ab9278ee43 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee15 expref: 229 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.994149:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100443 00000100:00100000:1.0:1785220581.994155:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+229:2407:x1871938519963648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220581.994157:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007865692b/0xf62499ab9278ee43 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee15 expref: 229 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994162:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220581.994166:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd5cdb08/0xf62499ab9278ebb8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ecd3 expref: 229 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994173:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7281:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.994175:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd5cdb08/0xf62499ab9278ebb8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ecd3 expref: 228 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994176:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7281:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.994181:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994186:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd5cdb08/0xf62499ab9278ebb8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ecd3 expref: 228 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994190:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994195:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.994198:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220581.994201:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994208:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220581.994209:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994221:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220581.994231:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac547450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+227:2407:x1871938519963776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 117us (1480us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220581.994235:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+227:2407:x1871938519963648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220581.994238:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100442 00000100:00100000:1.0:1785220581.994242:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100443 00010000:00010000:0.0:1785220581.994272:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994280:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994289:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994295:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.994319:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220581.994335:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.994341:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994348:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.994351:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994359:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994370:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994377:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a527f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7281:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994391:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.994392:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.994394:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220581.994451:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1229:3652:x1871938510711040:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9810us (10106us total) trans 4295070080 rc 0/0 00000100:00100000:0.0:1785220581.994458:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284511 00010000:00010000:3.0:1785220581.994468:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.994477:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.994485:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.994492:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.994513:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220581.994587:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1229:3652:x1871938510710016:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11637us (12184us total) trans 4295070081 rc 0/0 00000100:00100000:3.0:1785220581.994594:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284510 00010000:00010000:1.0:1785220581.994846:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220581.994852:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994860:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220581.994862:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994869:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994879:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994890:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5278 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220581.994905:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.994907:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220581.994909:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220581.995420:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510712448:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.995440:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.995463:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510712576:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.995473:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220581.995496:0:14223:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0000_UUID:14223:1871938510712704:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220581.995508:0:14223:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220581.996224:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac9c00/2, rc 0, transno 4295070080 00000100:00100000:0.0:1785220581.996232:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510711040:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220581.996243:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7dc00/2, rc 0, transno 4295070081 00000100:00100000:0.0:1785220581.996247:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510710016:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220581.996581:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220581.996596:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510714496:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.996652:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510714368:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.996693:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.996715:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220581.996724:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220581.996730:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510714240:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220581.996750:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220581.996849:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510714368 00000100:00100000:2.0:1785220581.996856:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510714368 00000100:00100000:2.0:1785220581.996898:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510714496 00000100:00100000:2.0:1785220581.996902:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510714496 00000100:00100000:3.0:1785220581.996910:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510714240 00000100:00100000:3.0:1785220581.996914:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510714240 00000100:00100000:2.0:1785220581.996922:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284524 00000100:00100000:2.0:1785220581.996929:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1231:3653:x1871938510714368:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220581.996945:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284525 00000100:00100000:3.0:1785220581.996952:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1231:3655:x1871938510714496:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220581.996954:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29318 count 1 00002000:00080000:3.0:1785220581.996961:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29319 count 1 00010000:00010000:3.0:1785220581.996976:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.996976:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.996994:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b1ecf720/0xf62499ab9278f209 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efe3 expref: 225 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.996994:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000294eeb94/0xf62499ab9278f1a0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efa4 expref: 225 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.997004:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7287:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220581.997004:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7286:0x0].0x0 since=23s v=(0 0) 00010000:00010000:3.0:1785220581.997009:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7287:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220581.997009:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7286:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220581.997014:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.997014:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.997039:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b1ecf720/0xf62499ab9278f209 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efe3 expref: 225 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.997039:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000294eeb94/0xf62499ab9278f1a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efa4 expref: 225 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220581.997052:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000294eeb94/0xf62499ab9278f1a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efa4 expref: 225 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220581.997053:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b1ecf720/0xf62499ab9278f209 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efe3 expref: 225 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220581.997065:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510714752:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.997065:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510714624:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220581.997276:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284526 00000100:00100000:0.0:1785220581.997286:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1231:3653:x1871938510714240:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220581.997299:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29317 count 1 00010000:00010000:0.0:1785220581.997312:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.997324:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009fdea818/0xf62499ab9278f0c7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef50 expref: 225 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.997331:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7285:0x0].0x0 since=23s v=(0 0) 00010000:00010000:0.0:1785220581.997334:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7285:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220581.997338:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.997356:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009fdea818/0xf62499ab9278f0c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef50 expref: 225 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220581.997366:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009fdea818/0xf62499ab9278f0c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef50 expref: 225 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220581.997381:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510714880:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220581.997470:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.000220:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 10 seconds 00000100:00100000:2.0:1785220582.000297:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.000335:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.000359:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519963904 00000100:00100000:2.0:1785220582.000367:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519963904 00010000:00010000:2.0:1785220582.000413:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006b074804/0xf62499ab9278ea45 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec40 expref: 226 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.000448:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964160 00000100:00100000:2.0:1785220582.000452:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964160 00010000:00010000:2.0:1785220582.000466:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006fa01f0c/0xf62499ab9279035f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f363 expref: 197 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.000490:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100444 00000100:00100000:2.0:1785220582.000499:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+226:2406:x1871938519963904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.000511:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.000516:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006b074804/0xf62499ab9278ea45 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec40 expref: 226 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000530:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006b074804/0xf62499ab9278ea45 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec40 expref: 226 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000552:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b074804/0xf62499ab9278ea45 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ec40 expref: 226 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000569:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.000573:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x727d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.000578:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000587:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000623:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.000655:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+225:2406:x1871938519963904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 156us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.000662:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100444 00000100:00100000:2.0:1785220582.000680:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100445 00000100:00100000:2.0:1785220582.000686:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2406:x1871938519964160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.000694:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.000699:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006fa01f0c/0xf62499ab9279035f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f363 expref: 197 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000708:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006fa01f0c/0xf62499ab9279035f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f363 expref: 197 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000722:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006fa01f0c/0xf62499ab9279035f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f363 expref: 197 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.000732:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.000736:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.000740:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001034:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001057:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.001080:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+196:2406:x1871938519964160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 393us (745us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.001114:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100445 00000100:00080000:2.0:1785220582.001145:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1785220582.001159:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001168:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001177:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001185:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001203:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.001367:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1232:3652:x1871938510709888:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18454us (19032us total) trans 4295070082 rc 0/0 00000100:00100000:2.0:1785220582.001377:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284509 00010000:00010000:2.0:1785220582.001399:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.001405:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001413:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.001418:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001427:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001440:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001448:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5271 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x727d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001466:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.001467:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.001469:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.001501:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.001526:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.001581:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.001600:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.001629:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964032 00000100:00100000:2.0:1785220582.001634:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964032 00010000:00010000:2.0:1785220582.001654:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000025f72b10/0xf62499ab9278eec1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee69 expref: 225 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.001658:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964288 00000100:00100000:3.0:1785220582.001662:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964288 00000100:00100000:2.0:1785220582.001679:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964416 00010000:00010000:3.0:1785220582.001680:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d07e59f7/0xf62499ab9278ed4e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edba expref: 226 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.001681:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964416 00010000:00010000:2.0:1785220582.001691:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009be89994/0xf62499ab92790095 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f31d expref: 196 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.001704:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964544 00000100:00100000:3.0:1785220582.001706:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964544 00000100:00100000:2.0:1785220582.001708:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100446 00000100:00100000:2.0:1785220582.001714:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+226:2406:x1871938519964032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.001717:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f2f47ec9/0xf62499ab927902e8 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f355 expref: 197 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001723:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.001727:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000025f72b10/0xf62499ab9278eec1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee69 expref: 226 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.001734:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100447 00010000:00010000:2.0:1785220582.001737:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000025f72b10/0xf62499ab9278eec1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee69 expref: 226 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.001740:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+226:2406:x1871938519964288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.001747:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.001751:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025f72b10/0xf62499ab9278eec1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ee69 expref: 226 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001752:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d07e59f7/0xf62499ab9278ed4e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edba expref: 226 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001773:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d07e59f7/0xf62499ab9278ed4e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edba expref: 226 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001786:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d07e59f7/0xf62499ab9278ed4e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7edba expref: 226 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001796:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7280:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.001800:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7280:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.001805:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001813:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001850:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.001872:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+225:2406:x1871938519964288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 132us (344us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.001878:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100447 00000100:00100000:3.0:1785220582.001893:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100448 00000100:00100000:3.0:1785220582.001899:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2406:x1871938519964416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.001907:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.001912:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009be89994/0xf62499ab92790095 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f31d expref: 197 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001921:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009be89994/0xf62499ab92790095 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f31d expref: 197 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001933:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.001936:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.001940:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001948:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.001960:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.001976:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2406:x1871938519964416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (396us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.001980:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7282:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.001983:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7282:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.001986:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100448 00010000:00010000:2.0:1785220582.001988:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.001996:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.001999:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100449 00000100:00100000:3.0:1785220582.002005:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+196:2406:x1871938519964544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.002012:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.002017:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f2f47ec9/0xf62499ab927902e8 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f355 expref: 196 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.002019:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.002026:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f2f47ec9/0xf62499ab927902e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f355 expref: 196 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.002034:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+223:2406:x1871938519964032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 321us (534us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.002037:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.002040:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ad:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220582.002041:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100446 00010000:00010000:3.0:1785220582.002044:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.002051:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.002062:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.002077:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+196:2406:x1871938519964544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 72us (476us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.002114:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100449 00000100:00100000:2.0:1785220582.002745:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.003011:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.003031:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.003044:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.003059:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964672 00000100:00100000:2.0:1785220582.003063:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964672 00010000:00010000:2.0:1785220582.003082:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c3d46ee1/0xf62499ab9278ff30 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f301 expref: 196 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.003229:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510713600:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.003288:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.003336:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964928 00000100:00100000:2.0:1785220582.003340:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964928 00010000:00010000:2.0:1785220582.003356:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000df1d902f/0xf62499ab92790017 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f30f expref: 197 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.003378:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519964800 00000100:00100000:2.0:1785220582.003381:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519964800 00010000:00010000:2.0:1785220582.003393:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004e3ea216/0xf62499ab927901b4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f339 expref: 198 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.003409:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965056 00000100:00100000:2.0:1785220582.003411:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965056 00010000:00010000:2.0:1785220582.003421:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000be6cf53b/0xf62499ab92790278 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f347 expref: 199 pid: 14204 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.003443:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100450 00000100:00100000:2.0:1785220582.003451:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2406:x1871938519964672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.003461:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220582.003466:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100451 00010000:00010000:2.0:1785220582.003466:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c3d46ee1/0xf62499ab9278ff30 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f301 expref: 199 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.003473:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2406:x1871938519964928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.003477:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c3d46ee1/0xf62499ab9278ff30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f301 expref: 199 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003480:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.003484:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000df1d902f/0xf62499ab92790017 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f30f expref: 199 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003492:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000df1d902f/0xf62499ab92790017 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f30f expref: 199 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.003494:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.003498:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.003502:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.003502:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003505:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.003509:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.003511:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003517:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.003524:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.003526:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.003550:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2406:x1871938519964672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (801us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.003556:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100450 00000100:00100000:2.0:1785220582.003574:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100452 00000100:00100000:2.0:1785220582.003581:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+198:2406:x1871938519964800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.003584:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+198:2406:x1871938519964928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (574us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.003590:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100451 00010000:00010000:2.0:1785220582.003590:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.003595:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004e3ea216/0xf62499ab927901b4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f339 expref: 198 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.003603:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100453 00010000:00010000:2.0:1785220582.003604:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004e3ea216/0xf62499ab927901b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f339 expref: 197 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.003609:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2406:x1871938519965056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.003619:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.003623:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000be6cf53b/0xf62499ab92790278 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f347 expref: 197 pid: 14204 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003632:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000be6cf53b/0xf62499ab92790278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f347 expref: 197 pid: 14204 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003644:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be6cf53b/0xf62499ab92790278 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f347 expref: 197 pid: 14204 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003655:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.003658:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.003662:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003670:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.003684:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.003687:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.003687:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.003692:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.003699:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.003701:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+196:2406:x1871938519965056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (656us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.003707:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100453 00010000:00010000:2.0:1785220582.003710:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.003722:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+195:2406:x1871938519964800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 143us (690us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.003725:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.003728:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100452 00010000:00010000:3.0:1785220582.003734:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003744:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003751:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.003779:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.003975:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.003992:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.004000:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+37:3654:x1871938510712064:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12449us (13751us total) trans 4295070658 rc 0/0 00000100:00100000:2.0:1785220582.004005:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965184 00000100:00100000:3.0:1785220582.004007:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284521 00000100:00100000:2.0:1785220582.004007:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965184 00010000:00010000:2.0:1785220582.004024:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c1396b89/0xf62499ab9278ef62 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eea8 expref: 223 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.004037:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510713856:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.004041:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965440 00000100:00100000:2.0:1785220582.004043:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965440 00010000:00010000:2.0:1785220582.004053:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000294eeb94/0xf62499ab9278f1a0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efa4 expref: 224 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.004055:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.004069:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100454 00000100:00100000:3.0:1785220582.004073:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100455 00000100:00100000:2.0:1785220582.004075:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+224:2406:x1871938519965184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.004079:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+224:2406:x1871938519965440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.004083:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.004112:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c1396b89/0xf62499ab9278ef62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eea8 expref: 224 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004122:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c1396b89/0xf62499ab9278ef62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eea8 expref: 224 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004201:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1396b89/0xf62499ab9278ef62 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eea8 expref: 224 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004212:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7283:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.004214:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7283:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.004217:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004225:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004242:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.004258:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+223:2406:x1871938519965184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 183us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.004264:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100454 00010000:00010000:2.0:1785220582.004282:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004290:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.004313:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.004319:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000294eeb94/0xf62499ab9278f1a0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efa4 expref: 222 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.004330:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000294eeb94/0xf62499ab9278f1a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efa4 expref: 222 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.004349:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7286:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.004352:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7286:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.004356:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004358:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.004365:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004366:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.004378:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.004381:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.004392:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2406:x1871938519965440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 313us (400us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.004398:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100455 00000004:00080000:3.0:1785220582.004435:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb67b80/2, rc 0, transno 4295070658 00000100:00100000:3.0:1785220582.004446:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510712064:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.004479:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1232:3654:x1871938510713728:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12021us (12391us total) trans 4295070083 rc 0/0 00000100:00100000:2.0:1785220582.004485:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284522 00000004:00080000:3.0:1785220582.004490:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac9180/2, rc 0, transno 4295070083 00000100:00100000:3.0:1785220582.004499:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510713728:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.004571:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510715008:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.004586:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.004601:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965312 00000100:00100000:3.0:1785220582.004602:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.004604:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965312 00010000:00010000:2.0:1785220582.004684:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000903ea35a/0xf62499ab9278f011 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eef5 expref: 222 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.004706:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100456 00000100:00100000:2.0:1785220582.004714:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9af050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2406:x1871938519965312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.004725:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.004729:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000903ea35a/0xf62499ab9278f011 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eef5 expref: 222 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004740:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000903ea35a/0xf62499ab9278f011 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eef5 expref: 222 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004812:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7284:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.004815:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7284:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.004819:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004917:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.004927:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.004939:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9af050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2406:x1871938519965312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 227us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.004943:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100456 00000100:00100000:2.0:1785220582.004976:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.005008:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965568 00000100:00100000:3.0:1785220582.005012:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965568 00010000:00010000:3.0:1785220582.005029:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cb4b0d23/0xf62499ab92790128 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f32b expref: 195 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.005056:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100457 00010000:00010000:2.0:1785220582.005108:0:14257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000294eeb94/0xf62499ab9278f1a0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efa4 expref: 221 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.005131:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510714624:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.005145:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005155:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005162:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005170:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005177:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005193:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220582.005280:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+195:2406:x1871938519965568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220582.005282:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1232:3653:x1871938510714368:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8353us (8589us total) trans 4295070084 rc 0/0 00000100:00100000:2.0:1785220582.005288:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284524 00010000:00010000:3.0:1785220582.005290:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.005294:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cb4b0d23/0xf62499ab92790128 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f32b expref: 195 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.005299:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510715008 00000100:00100000:2.0:1785220582.005302:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510715008 00010000:00010000:3.0:1785220582.005304:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cb4b0d23/0xf62499ab92790128 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f32b expref: 195 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.005317:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.005320:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72aa:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220582.005321:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284527 00010000:00010000:3.0:1785220582.005325:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.005326:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1232:3655:x1871938510715008:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.005333:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.005344:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00080000:2.0:1785220582.005345:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29320 count 1 00000100:00100000:3.0:1785220582.005357:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+195:2406:x1871938519965568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.005363:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100457 00010000:00010000:2.0:1785220582.005363:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005375:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007bf3b9de/0xf62499ab9278f25d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f022 expref: 220 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005382:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7288:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220582.005386:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7288:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.005390:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005413:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007bf3b9de/0xf62499ab9278f25d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f022 expref: 220 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005425:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007bf3b9de/0xf62499ab9278f25d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f022 expref: 220 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005425:0:14248:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009be89994/0xf62499ab92790095 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f31d expref: 194 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.005437:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0000_UUID:14248:1871938510712832:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.005438:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510715136:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.005452:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005460:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005466:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.005467:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd777b40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.005475:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005482:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.005482:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005488:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005495:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.005500:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005509:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005529:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220582.005531:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005538:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a52c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005554:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.005558:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005564:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.005567:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005574:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005584:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005591:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a52cc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7283:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005605:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.005608:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.005611:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+37:3653:x1871938510711680:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19065us (20231us total) trans 4295070659 rc 0/0 00000100:00100000:1.0:1785220582.005617:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284515 00010000:00010000:2.0:1785220582.005628:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.005631:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005638:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005647:0:13908:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2f47ec9/0xf62499ab927902e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f355 expref: 193 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005648:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005656:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a52da lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7286:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.005664:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510712960:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.005669:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.005672:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005675:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005679:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.005682:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005684:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005691:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005691:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005699:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005701:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005707:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005709:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c158028/0xf62499ab927a529b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005721:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.005734:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.005799:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00000100:00100000:1.0:1785220582.005885:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+37:3653:x1871938510712192:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19188us (20469us total) trans 4295070660 rc 0/0 00000100:00100000:1.0:1785220582.005891:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284516 00010000:00010000:1.0:1785220582.005930:0:14208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3d46ee1/0xf62499ab9278ff30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f301 expref: 192 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.005941:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510713216:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.005953:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005961:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005968:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005976:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005978:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.005984:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005985:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.005989:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.005996:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006010:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006017:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d7fb77d/0xf62499ab927a52a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006030:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.006033:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006039:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.006043:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006051:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006061:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006072:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.006073:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.006075:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.006075:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.006167:0:14193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df1d902f/0xf62499ab92790017 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f30f expref: 191 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.006178:0:14193:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510713472:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.006190:0:14193:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006198:0:14193:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006205:0:14193:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006213:0:14193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.006220:0:14193:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.006228:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1785220582.006232:0:14208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c64f6865/0xf62499ab927a52b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.006324:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+37:3654:x1871938510711424:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15074us (16239us total) trans 4295070661 rc 0/0 00000100:00100000:1.0:1785220582.006331:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284518 00010000:00010000:3.0:1785220582.007524:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.007535:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.007546:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.007551:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.007561:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.007578:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.007586:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.007588:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.007590:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.007623:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04380/2, rc 0, transno 4295070661 00000100:00100000:3.0:1785220582.007630:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510711424:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.008243:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464780 going to sleep for 10 seconds 00000004:00080000:1.0:1785220582.008276:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7f100/2, rc 0, transno 4295070082 00000100:00100000:1.0:1785220582.008480:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510709888:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.008579:0:14193:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1785220582.008587:0:14193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075879d7f/0xf62499ab927a52be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.008713:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.008726:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510715264:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.008794:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.008929:0:13874:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510715264 00000100:00100000:1.0:1785220582.008935:0:13874:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510715264 00000100:00100000:2.0:1785220582.008957:0:14193:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+37:3654:x1871938510711552:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17436us (18743us total) trans 4295070662 rc 0/0 00000100:00100000:2.0:1785220582.008964:0:14193:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284520 00000100:00100000:1.0:1785220582.008974:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284528 00000100:00100000:1.0:1785220582.008982:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1233:3652:x1871938510715264:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.009002:0:13818:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e3ea216/0xf62499ab927901b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f339 expref: 190 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.009006:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29321 count 1 00000100:00100000:2.0:1785220582.009014:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510713344:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.009027:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009031:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009039:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.009043:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008bcc333e/0xf62499ab9278f295 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f045 expref: 220 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009047:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.009052:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7289:0x0].0x0 since=23s v=(0 0) 00010000:00010000:2.0:1785220582.009054:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.009056:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7289:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.009060:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009063:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009078:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.009111:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008bcc333e/0xf62499ab9278f295 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f045 expref: 220 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.009393:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.009401:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.009409:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.009413:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.009422:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.009437:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.009445:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2eba103/0xf62499ab927a52b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.009461:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.009463:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.009465:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.009549:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+37:3654:x1871938510711936:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17766us (19002us total) trans 4295070663 rc 0/0 00010000:00010000:1.0:1785220582.009550:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008bcc333e/0xf62499ab9278f295 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f045 expref: 220 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.009556:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284519 00000100:00100000:1.0:1785220582.009565:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510715392:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.009580:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009588:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009595:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009601:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.009616:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bad80 going to sleep for 11 seconds 00000004:00080000:3.0:1785220582.009670:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04700/2, rc 0, transno 4295070662 00000100:00100000:3.0:1785220582.009676:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510711552:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.009684:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1785220582.009686:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e05500/2, rc 0, transno 4295070663 00000100:00100000:3.0:1785220582.009689:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e05500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510711936:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.009924:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1234:3654:x1871938510710912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23804us (25433us total) trans 4295070085 rc 0/0 00000100:00100000:2.0:1785220582.009932:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284513 00000004:00080000:3.0:1785220582.009952:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac8a80/2, rc 0, transno 4295070085 00010000:00010000:2.0:1785220582.009957:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.009960:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510710912:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.009971:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009979:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009985:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.009997:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.010023:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.010028:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010036:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.010039:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010047:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010059:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.010066:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1234:3652:x1871938510711168:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24481us (25626us total) trans 4295070086 rc 0/0 00010000:00010000:1.0:1785220582.010067:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a77a8686/0xf62499ab927a528d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7280:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.010073:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284512 00010000:00010000:2.0:1785220582.010115:0:14223:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.010123:0:14223:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.010131:0:14223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.010138:0:14223:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.010152:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.010153:0:14223:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.010157:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.010164:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.010167:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.010175:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.010185:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.010193:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006135719f/0xf62499ab927a5286 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7282:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.010226:0:14223:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb67800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+37:3653:x1871938510712320:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23849us (24887us total) trans 4295070664 rc 0/0 00000100:00100000:2.0:1785220582.010233:0:14223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284514 00010000:00010000:1.0:1785220582.010237:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.010241:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010247:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.010248:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.010251:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010257:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010272:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010279:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e899db7/0xf62499ab927a5294 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010293:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.010295:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.010297:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.010314:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510714880:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.010329:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.010353:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94acb480/2, rc 0, transno 4295070086 00000100:00100000:1.0:1785220582.010363:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510711168:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.010373:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.010379:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510715520:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.010408:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.010461:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.010469:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510715648:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.010494:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.010512:0:14250:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510715520 00000100:00100000:2.0:1785220582.010515:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510715136:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.010516:0:14250:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510715520 00000100:00100000:1.0:1785220582.010540:0:14250:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510715648 00000100:00100000:1.0:1785220582.010544:0:14250:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510715648 00000100:00100000:1.0:1785220582.010561:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284529 00000100:00100000:1.0:1785220582.010568:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1235:3652:x1871938510715520:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.010570:0:14262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000903ea35a/0xf62499ab9278f011 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7eef5 expref: 220 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.010584:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29322 count 1 00000100:00100000:0.0:1785220582.010587:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510714112:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.010599:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.010599:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.010607:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010612:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b92ef790/0xf62499ab9278f2cd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f07d expref: 219 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010620:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x728a:0x0].0x0 since=23s v=(0 0) 00010000:00010000:1.0:1785220582.010623:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.010627:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010649:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b92ef790/0xf62499ab9278f2cd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f07d expref: 219 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.010660:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b92ef790/0xf62499ab9278f2cd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f07d expref: 219 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.010672:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510715776:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.010681:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.010689:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.010696:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.010708:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.010709:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.010976:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1236:3654:x1871938510713984:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17795us (18308us total) trans 4295070087 rc 0/0 00000100:00100000:0.0:1785220582.010983:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284523 00000100:00100000:0.0:1785220582.010998:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284530 00000100:00100000:0.0:1785220582.011005:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1235:3655:x1871938510715648:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.011017:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29323 count 1 00010000:00010000:0.0:1785220582.011029:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011040:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008e429ba5/0xf62499ab9278f30c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0bc expref: 219 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011046:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x728b:0x0].0x0 since=22s v=(0 0) 00010000:00010000:0.0:1785220582.011050:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.011053:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011071:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008e429ba5/0xf62499ab9278f30c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0bc expref: 219 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011079:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008e429ba5/0xf62499ab9278f30c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0bc expref: 219 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.011105:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510715904:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.011130:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464cc0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.011157:0:14259:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb4b0d23/0xf62499ab92790128 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f32b expref: 189 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.011168:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510713088:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.011180:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011193:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011199:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011207:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011213:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011227:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.011304:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+37:3653:x1871938510711808:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20292us (26005us total) trans 4295070665 rc 0/0 00000100:00100000:0.0:1785220582.011311:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284517 00010000:00010000:0.0:1785220582.011328:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.011334:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011341:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.011345:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011353:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011366:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011372:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a52d3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011386:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.011390:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011397:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.011400:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011408:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011418:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011426:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a52a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.011438:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.011440:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.011443:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.011454:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.011522:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a86a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510716032:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.011545:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.011565:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.011570:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510716416:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.011590:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.011611:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510716032 00000100:00100000:1.0:1785220582.012071:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510716416 00000100:00100000:1.0:1785220582.012077:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510716416 00000100:00100000:1.0:1785220582.012152:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284531 00000100:00100000:1.0:1785220582.012160:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+37:3655:x1871938510716416:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.012179:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29362 count 1 00010000:00010000:1.0:1785220582.012194:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.012207:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c1011d1c/0xf62499ab92790653 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f39b expref: 188 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.012215:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b2:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.012218:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.012222:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.012244:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c1011d1c/0xf62499ab92790653 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f39b expref: 188 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.012255:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c1011d1c/0xf62499ab92790653 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f39b expref: 188 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.012266:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510716544:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.012273:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510716032 00000100:00100000:0.0:1785220582.012308:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284532 00000100:00100000:1.0:1785220582.012312:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.012315:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+38:3655:x1871938510716032:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.012327:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29359 count 1 00010000:00010000:0.0:1785220582.012340:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.012350:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000353150f2/0xf62499ab927903f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f371 expref: 188 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.012356:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72af:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220582.012359:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.012362:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.012378:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000353150f2/0xf62499ab927903f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f371 expref: 188 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.012386:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000353150f2/0xf62499ab927903f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f371 expref: 188 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.012396:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510716672:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.012400:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510716160:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.012419:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.012426:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.012646:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.012652:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510716288:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.012675:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.013080:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.014333:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510716160 00000100:00100000:0.0:1785220582.014347:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510716160 00000100:00100000:0.0:1785220582.014391:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284533 00000100:00100000:0.0:1785220582.014401:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+39:3652:x1871938510716160:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.014429:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29360 count 1 00010000:00010000:0.0:1785220582.014449:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.014466:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000095af4cd4/0xf62499ab92790503 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f37f expref: 188 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.014474:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b0:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.014478:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.014481:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.014508:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000095af4cd4/0xf62499ab92790503 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f37f expref: 188 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.014515:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.014519:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000095af4cd4/0xf62499ab92790503 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f37f expref: 188 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.014539:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510716800:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.014546:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.014589:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000004:00080000:1.0:1785220582.014658:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7d500/2, rc 0, transno 4295070087 00000100:00100000:1.0:1785220582.014670:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510713984:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.014721:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.014729:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510716928:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.014767:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.014789:0:14175:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510716288 00000100:00100000:1.0:1785220582.014792:0:14175:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510716288 00000100:00100000:1.0:1785220582.014816:0:14175:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510716928 00000100:00100000:1.0:1785220582.014819:0:14175:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510716928 00000100:00100000:0.0:1785220582.014903:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.014957:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284534 00000100:00100000:2.0:1785220582.014966:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+40:3652:x1871938510716288:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.014982:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29361 count 1 00010000:00010000:2.0:1785220582.014995:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015007:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002772fbb2/0xf62499ab927905dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f38d expref: 188 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015013:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b1:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.015015:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.015020:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015038:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002772fbb2/0xf62499ab927905dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f38d expref: 188 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015047:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002772fbb2/0xf62499ab927905dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f38d expref: 188 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.015057:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510717056:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.015202:0:14175:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284535 00000100:00100000:1.0:1785220582.015209:0:14175:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1236:3654:x1871938510716928:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.015221:0:14175:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29324 count 1 00010000:00010000:1.0:1785220582.015234:0:14175:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.015246:0:14175:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000abc71b06/0xf62499ab9278f344 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0f4 expref: 219 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.015252:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x728c:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220582.015255:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.015259:0:14175:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.015279:0:14175:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000abc71b06/0xf62499ab9278f344 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0f4 expref: 219 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.015295:0:14175:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000abc71b06/0xf62499ab9278f344 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0f4 expref: 219 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.015306:0:14175:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510717184:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.015354:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01780 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.015375:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.015385:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965824 00000100:00100000:2.0:1785220582.015389:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965824 00010000:00010000:2.0:1785220582.015408:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007bf3b9de/0xf62499ab9278f25d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f022 expref: 220 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.015436:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965696 00000100:00100000:2.0:1785220582.015439:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965696 00010000:00010000:2.0:1785220582.015452:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009fdea818/0xf62499ab9278f0c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef50 expref: 221 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.015469:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519965952 00000100:00100000:2.0:1785220582.015471:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519965952 00010000:00010000:2.0:1785220582.015483:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b1ecf720/0xf62499ab9278f209 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efe3 expref: 222 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.015500:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100458 00000100:00100000:2.0:1785220582.015507:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+222:2407:x1871938519965824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.015516:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.015521:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007bf3b9de/0xf62499ab9278f25d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f022 expref: 222 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015531:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007bf3b9de/0xf62499ab9278f25d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f022 expref: 222 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015550:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007bf3b9de/0xf62499ab9278f25d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f022 expref: 222 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015563:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7288:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.015565:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7288:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.015570:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015578:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.015598:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.016052:0:14175:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.016597:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+221:2407:x1871938519965824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1087us (2079us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.016607:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100458 00000100:00100000:2.0:1785220582.016646:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100459 00000100:00100000:2.0:1785220582.016656:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+220:2407:x1871938519965696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.016666:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.016672:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009fdea818/0xf62499ab9278f0c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef50 expref: 220 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.016689:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009fdea818/0xf62499ab9278f0c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef50 expref: 220 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.016705:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fdea818/0xf62499ab9278f0c7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ef50 expref: 220 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.016718:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7285:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.016722:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7285:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.016726:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.016734:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.016766:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.016791:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+219:2407:x1871938519965696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 137us (2242us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.016798:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100459 00000100:00100000:2.0:1785220582.016812:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100460 00000100:00100000:2.0:1785220582.016819:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a84380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+218:2407:x1871938519965952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.016979:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.016989:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.016999:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017007:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017023:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.017208:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1237:3653:x1871938510714240:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19923us (20455us total) trans 4295070088 rc 0/0 00000100:00100000:1.0:1785220582.017215:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284526 00010000:00010000:2.0:1785220582.017225:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.017231:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b1ecf720/0xf62499ab9278f209 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efe3 expref: 218 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017246:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b1ecf720/0xf62499ab9278f209 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efe3 expref: 218 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017260:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7287:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.017263:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7287:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.017267:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017274:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017287:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.017306:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a84380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+218:2407:x1871938519965952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 488us (2405us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.017313:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100460 00010000:00010000:2.0:1785220582.017359:0:14252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b1ecf720/0xf62499ab9278f209 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efe3 expref: 217 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.017370:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510714752:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.017391:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017400:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017415:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017423:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017431:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017450:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.017499:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.017505:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017519:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.017522:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.017528:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1237:3655:x1871938510714496:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20577us (20812us total) trans 4295070089 rc 0/0 00010000:00010000:1.0:1785220582.017532:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.017535:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284525 00010000:00010000:1.0:1785220582.017544:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017552:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a52e8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017558:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017565:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017568:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.017571:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017574:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017578:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.017582:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017583:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017590:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017599:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.017602:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.017606:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a52e1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7287:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017619:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.017623:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017629:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.017632:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017638:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017648:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017655:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a52ef lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7288:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.017668:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.017670:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.017672:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.017742:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1237:3655:x1871938510715008:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12415us (13139us total) trans 4295070090 rc 0/0 00000100:00100000:2.0:1785220582.017748:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284527 00010000:00010000:2.0:1785220582.017773:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.017960:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.017962:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:13874:1871938510715392:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.017976:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.017982:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbe300/2, rc 0, transno 4295070090 00000100:00100000:0.0:1785220582.017993:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510715008:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.018049:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a86680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510717312:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.018102:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.018252:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbca80/2, rc 0, transno 4295070089 00000100:00100000:0.0:1785220582.018261:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510714496:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.018289:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220582.018296:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510717440:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.018320:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.018856:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510717312 00000100:00100000:2.0:1785220582.018863:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510717312 00000100:00100000:2.0:1785220582.018898:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510717440 00000100:00100000:2.0:1785220582.018901:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510717440 00000100:00100000:1.0:1785220582.018910:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284536 00000100:00100000:1.0:1785220582.018920:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9573fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1238:3652:x1871938510717312:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.018928:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284537 00000100:00100000:2.0:1785220582.018933:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1238:3652:x1871938510717440:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.018944:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29326 count 1 00002000:00080000:1.0:1785220582.018944:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29325 count 1 00010000:00010000:2.0:1785220582.018963:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.018963:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.018980:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000053751f2b/0xf62499ab9278f3ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f15d expref: 216 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.018980:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c679e409/0xf62499ab9278f383 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f133 expref: 216 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.018989:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x728e:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220582.018989:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x728d:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220582.018993:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.018993:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.018999:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.018999:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.019023:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000053751f2b/0xf62499ab9278f3ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f15d expref: 216 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.019023:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c679e409/0xf62499ab9278f383 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f133 expref: 216 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.019035:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c679e409/0xf62499ab9278f383 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f133 expref: 216 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.019037:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000053751f2b/0xf62499ab9278f3ad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f15d expref: 216 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.019048:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510717568:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.019048:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510717696:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.019278:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bad80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.019525:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.019718:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.019817:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510715776:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.019836:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.019886:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966080 00000100:00100000:2.0:1785220582.019891:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966080 00010000:00010000:2.0:1785220582.019913:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008bcc333e/0xf62499ab9278f295 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f045 expref: 217 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.019946:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100461 00000100:00100000:2.0:1785220582.019954:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+217:2407:x1871938519966080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.019967:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.019973:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008bcc333e/0xf62499ab9278f295 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f045 expref: 217 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.019984:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008bcc333e/0xf62499ab9278f295 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f045 expref: 217 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.020004:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008bcc333e/0xf62499ab9278f295 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f045 expref: 217 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.020018:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7289:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.020022:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7289:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.020026:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.020035:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.020065:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.020115:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+216:2407:x1871938519966080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 161us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.020122:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100461 00010000:00010000:0.0:1785220582.020181:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020191:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020205:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020213:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020230:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.020374:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+1239:3652:x1871938510715264:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11392us (11578us total) trans 4295070091 rc 0/0 00000100:00100000:0.0:1785220582.020381:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284528 00010000:00010000:0.0:1785220582.020400:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000004:00080000:1.0:1785220582.020406:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c0a80/2, rc 0, transno 4295070091 00010000:00010000:0.0:1785220582.020406:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020414:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735400 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:1.0:1785220582.020416:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510715264:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.020419:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020427:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020440:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020446:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091eec52a/0xf62499ab927a52f6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7289:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.020466:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.020468:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.020470:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.020525:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.020536:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510717824:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.020565:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.020936:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510717824 00000100:00100000:2.0:1785220582.020942:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510717824 00000100:00100000:2.0:1785220582.020979:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284538 00000100:00100000:2.0:1785220582.020987:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94acb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1239:3655:x1871938510717824:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.021005:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29327 count 1 00010000:00010000:2.0:1785220582.021024:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.021038:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000051db8067/0xf62499ab9278f424 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b1 expref: 215 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.021046:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x728f:0x0].0x0 since=22s v=(0 0) 00010000:00010000:2.0:1785220582.021049:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.021053:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.021081:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000051db8067/0xf62499ab9278f424 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b1 expref: 215 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.021133:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000051db8067/0xf62499ab9278f424 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b1 expref: 215 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.021146:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510717952:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.021194:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.022153:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.022234:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966208 00000100:00100000:0.0:1785220582.022241:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966208 00010000:00010000:0.0:1785220582.022268:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b92ef790/0xf62499ab9278f2cd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f07d expref: 216 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.022304:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100462 00000100:00100000:0.0:1785220582.022313:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+216:2407:x1871938519966208:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.022324:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.022329:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b92ef790/0xf62499ab9278f2cd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f07d expref: 216 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.022341:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b92ef790/0xf62499ab9278f2cd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f07d expref: 216 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.022361:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b92ef790/0xf62499ab9278f2cd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f07d expref: 216 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.022382:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.022386:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.022391:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.022400:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.022430:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.022464:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+215:2407:x1871938519966208:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 153us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.022471:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100462 00010000:00010000:3.0:1785220582.022515:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.022518:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.022521:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.022575:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbe680/2, rc 0, transno 4295070088 00000100:00100000:3.0:1785220582.022595:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510714240:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.022610:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbfb80/2, rc 0, transno 4295070084 00000100:00100000:3.0:1785220582.022613:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510714368:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.022627:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e04000/2, rc 0, transno 4295070659 00000100:00100000:3.0:1785220582.022635:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e04000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510711680:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.022644:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64380/2, rc 0, transno 4295070660 00000100:00100000:3.0:1785220582.022647:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510712192:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.022657:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e06a00/2, rc 0, transno 4295070665 00000100:00100000:3.0:1785220582.022660:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e06a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510711808:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.022668:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64e00/2, rc 0, transno 4295070664 00000100:00100000:3.0:1785220582.022672:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510712320:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.022720:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.022730:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.022739:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.022747:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.022778:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.022808:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.022825:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.022843:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.022860:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966336 00000100:00100000:2.0:1785220582.022864:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966336 00010000:00010000:2.0:1785220582.022882:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008e429ba5/0xf62499ab9278f30c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0bc expref: 215 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.022903:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966592 00000100:00100000:2.0:1785220582.022905:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966592 00010000:00010000:2.0:1785220582.022918:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000353150f2/0xf62499ab927903f9 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f371 expref: 189 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.022933:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966464 00000100:00100000:1.0:1785220582.022934:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1240:3652:x1871938510715520:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12365us (12525us total) trans 4295070092 rc 0/0 00000100:00100000:2.0:1785220582.022935:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966464 00000100:00100000:1.0:1785220582.022942:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284529 00010000:00010000:2.0:1785220582.022945:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c1011d1c/0xf62499ab92790653 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f39b expref: 190 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.022963:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100463 00000100:00100000:2.0:1785220582.022969:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90702050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+215:2407:x1871938519966336:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.022971:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510716544:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.022976:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.022980:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008e429ba5/0xf62499ab9278f30c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0bc expref: 215 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.022987:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.022990:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008e429ba5/0xf62499ab9278f30c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0bc expref: 215 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023003:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.023007:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728b:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.023007:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100464 00010000:00010000:2.0:1785220582.023012:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.023013:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+190:2407:x1871938519966592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.023020:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.023021:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.023026:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000353150f2/0xf62499ab927903f9 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f371 expref: 190 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023031:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.023035:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000353150f2/0xf62499ab927903f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f371 expref: 190 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.023049:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72af:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220582.023052:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90702050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+215:2407:x1871938519966336:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (245us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.023052:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.023056:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.023059:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100463 00010000:00010000:1.0:1785220582.023064:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.023074:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.023114:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+190:2407:x1871938519966592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.023116:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100465 00000100:00100000:1.0:1785220582.023120:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100464 00000100:00100000:2.0:1785220582.023134:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+189:2407:x1871938519966464:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.023144:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.023148:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c1011d1c/0xf62499ab92790653 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f39b expref: 189 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.023150:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c1880/2, rc 0, transno 4295070092 00000100:00100000:1.0:1785220582.023154:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510715520:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.023158:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c1011d1c/0xf62499ab92790653 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f39b expref: 189 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023171:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1011d1c/0xf62499ab92790653 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f39b expref: 189 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023182:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.023185:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.023189:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023197:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.023205:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.023218:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.023224:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.023232:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.023236:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+188:2407:x1871938519966464:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.023242:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100465 00010000:00010000:1.0:1785220582.023246:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.023253:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966720 00010000:00010000:1.0:1785220582.023254:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.023255:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966720 00010000:00010000:1.0:1785220582.023265:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.023267:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000095af4cd4/0xf62499ab92790503 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f37f expref: 188 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.023289:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100466 00000100:00100000:2.0:1785220582.023295:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+188:2407:x1871938519966720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.023301:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.023305:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000095af4cd4/0xf62499ab92790503 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f37f expref: 188 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023314:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000095af4cd4/0xf62499ab92790503 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f37f expref: 188 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023326:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.023328:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.023332:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023339:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023350:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.023355:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+41:3655:x1871938510716416:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11196us (11764us total) trans 4295070666 rc 0/0 00000100:00100000:1.0:1785220582.023361:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284531 00000100:00100000:2.0:1785220582.023364:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+188:2407:x1871938519966720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 70us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.023370:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100466 00000100:00100000:2.0:1785220582.023469:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0000_UUID:14247:1871938510717056:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.023483:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.023512:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.023527:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966848 00000100:00100000:2.0:1785220582.023530:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966848 00010000:00010000:2.0:1785220582.023544:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002772fbb2/0xf62499ab927905dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f38d expref: 188 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.023565:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100467 00000100:00100000:2.0:1785220582.023571:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+188:2407:x1871938519966848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.023579:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.023583:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002772fbb2/0xf62499ab927905dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f38d expref: 188 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023592:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002772fbb2/0xf62499ab927905dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f38d expref: 188 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023604:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002772fbb2/0xf62499ab927905dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f38d expref: 188 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023740:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.023747:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023765:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.023767:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.023770:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023771:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.023775:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023779:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023784:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023791:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023798:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa6fec17/0xf62499ab927a52fd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023800:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.023816:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1785220582.023821:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+187:2407:x1871938519966848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 250us (309us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.023827:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100467 00010000:00010000:3.0:1785220582.023833:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023840:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.023843:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023844:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023853:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023857:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023861:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023867:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023869:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023875:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2d0c599/0xf62499ab927a530b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.023882:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.023888:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.023891:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023897:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.023900:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023907:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023916:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023923:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a5320 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.023940:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.023942:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.023944:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.023963:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+41:3652:x1871938510716288:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8999us (11287us total) trans 4295070667 rc 0/0 00000100:00100000:2.0:1785220582.023971:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284534 00010000:00010000:2.0:1785220582.023989:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.024184:0:14262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008e429ba5/0xf62499ab9278f30c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0bc expref: 214 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.024197:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0001_UUID:14262:1871938510715904:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.024213:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024222:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024230:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024238:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024251:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024273:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.024357:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1785220582.024362:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+1240:3655:x1871938510715648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13358us (13867us total) trans 4295070093 rc 0/0 00010000:00010000:3.0:1785220582.024363:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.024369:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284530 00010000:00010000:3.0:1785220582.024371:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.024374:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.024381:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.024392:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.024399:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a5304 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024400:0:14159:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000353150f2/0xf62499ab927903f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f371 expref: 186 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.024411:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510716672:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.024414:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.024415:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.024417:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.024422:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024431:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024438:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024446:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024453:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024473:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.024478:0:14175:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510717184:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.024490:0:14175:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.024520:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a87100/2, rc 0, transno 4295070667 00000100:00100000:1.0:1785220582.024526:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510716288:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.024544:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+41:3655:x1871938510716032:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12229us (12998us total) trans 4295070668 rc 0/0 00000100:00100000:0.0:1785220582.024550:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284532 00010000:00010000:0.0:1785220582.024566:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.024571:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.024578:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.024579:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.024582:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024590:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024601:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024607:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a5312 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024634:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.024636:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.024637:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.024719:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.024732:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510718080:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.024776:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.024803:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718080 00010000:00010000:0.0:1785220582.024807:0:14177:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095af4cd4/0xf62499ab92790503 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f37f expref: 185 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.024808:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718080 00000100:00100000:0.0:1785220582.024818:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510716800:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.024828:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024837:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024843:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024851:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024859:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024873:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.024882:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.024886:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024892:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.024895:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024907:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024919:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024927:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a5319 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.024941:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.024943:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718336:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.024943:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.024945:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.024971:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.024990:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718848:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.025009:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.025014:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+42:3652:x1871938510716160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10615us (12587us total) trans 4295070669 rc 0/0 00000100:00100000:0.0:1785220582.025021:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284533 00000100:00100000:3.0:1785220582.025025:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1785220582.025029:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718208:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.025032:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718336 00000100:00100000:0.0:1785220582.025035:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718336 00000100:00100000:3.0:1785220582.025049:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.025059:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718848 00000100:00100000:0.0:1785220582.025061:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718848 00000100:00100000:3.0:1785220582.025063:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718720:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.025079:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718208 00000100:00100000:3.0:1785220582.025081:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.025081:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718208 00000100:00100000:0.0:1785220582.025118:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718720 00000100:00100000:0.0:1785220582.025121:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718720 00000100:00100000:0.0:1785220582.025140:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284539 00000100:00100000:0.0:1785220582.025148:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+43:3653:x1871938510718336:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.025167:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29365 count 1 00000100:00100000:1.0:1785220582.025171:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284540 00000100:00100000:1.0:1785220582.025177:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1241:3653:x1871938510718848:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.025184:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.025186:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29329 count 1 00010000:00010000:1.0:1785220582.025199:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025199:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d7558fe1/0xf62499ab92790821 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3c5 expref: 184 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025208:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b5:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.025210:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002456099a/0xf62499ab9278f455 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1cd expref: 213 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025211:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.025216:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025217:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7291:0x0].0x0 since=22s v=(0 0) 00010000:00010000:1.0:1785220582.025220:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7291:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.025223:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025238:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002456099a/0xf62499ab9278f455 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1cd expref: 213 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025238:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d7558fe1/0xf62499ab92790821 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3c5 expref: 184 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025250:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002456099a/0xf62499ab9278f455 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1cd expref: 213 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025250:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d7558fe1/0xf62499ab92790821 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3c5 expref: 184 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.025262:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510719104:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.025262:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510719232:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.025307:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bae40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.025325:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284541 00000100:00100000:1.0:1785220582.025332:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+43:3653:x1871938510718208:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.025343:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29364 count 1 00010000:00010000:1.0:1785220582.025354:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025365:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fde976c3/0xf62499ab9279079c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b7 expref: 184 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025372:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b4:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.025375:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.025378:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025397:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fde976c3/0xf62499ab9279079c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b7 expref: 184 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025406:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fde976c3/0xf62499ab9279079c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b7 expref: 184 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.025415:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510719360:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.025809:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.025816:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.025827:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284542 00000100:00100000:0.0:1785220582.025835:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1241:3653:x1871938510718720:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.025848:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29328 count 1 00010000:00010000:0.0:1785220582.025862:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025873:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000080047ebe/0xf62499ab9278f447 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1bf expref: 213 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025884:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7290:0x0].0x0 since=22s v=(0 0) 00010000:00010000:0.0:1785220582.025886:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7290:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.025889:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025909:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000080047ebe/0xf62499ab9278f447 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1bf expref: 213 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.025918:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000080047ebe/0xf62499ab9278f447 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1bf expref: 213 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.025921:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284543 00000100:00100000:0.0:1785220582.025928:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510719488:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.025930:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+43:3654:x1871938510718080:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.025942:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29363 count 1 00000100:00100000:0.0:1785220582.025951:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.025953:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025964:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000054ba2e46/0xf62499ab927906e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a9 expref: 184 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.025971:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b3:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.025973:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.025977:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.025980:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510718592:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.025995:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000054ba2e46/0xf62499ab927906e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a9 expref: 184 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.026004:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000054ba2e46/0xf62499ab927906e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a9 expref: 184 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.026005:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.026014:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510719616:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.026027:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718592 00000100:00100000:0.0:1785220582.026031:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718592 00000100:00100000:1.0:1785220582.026034:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.026054:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510718464:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.026055:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284544 00000100:00100000:0.0:1785220582.026062:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+44:3655:x1871938510718592:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.026072:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29367 count 1 00000100:00100000:1.0:1785220582.026077:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.026082:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026114:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000592ea131/0xf62499ab927909a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e1 expref: 184 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.026122:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510718976:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.026133:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b7:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.026136:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.026140:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.026142:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.026159:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000592ea131/0xf62499ab927909a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e1 expref: 184 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026168:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000592ea131/0xf62499ab927909a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e1 expref: 184 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.026177:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510719744:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.026203:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.026217:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718464 00000100:00100000:0.0:1785220582.026220:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718464 00000100:00100000:0.0:1785220582.026242:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510718976 00000100:00100000:0.0:1785220582.026245:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510718976 00000100:00100000:0.0:1785220582.026262:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284545 00000100:00100000:0.0:1785220582.026267:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+45:3654:x1871938510718464:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.026279:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29366 count 1 00010000:00010000:0.0:1785220582.026289:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026300:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006dd11019/0xf62499ab927908c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3d3 expref: 184 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026311:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b6:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.026314:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.026317:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026335:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006dd11019/0xf62499ab927908c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3d3 expref: 184 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026344:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006dd11019/0xf62499ab927908c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3d3 expref: 184 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.026356:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510719872:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.026383:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.026400:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284546 00000100:00100000:0.0:1785220582.026407:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1242:3654:x1871938510718976:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.026419:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29330 count 1 00010000:00010000:0.0:1785220582.026431:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026442:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ce3d078d/0xf62499ab9278f46a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1db expref: 213 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026448:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7292:0x0].0x0 since=22s v=(0 0) 00010000:00010000:0.0:1785220582.026451:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7292:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.026455:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026473:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ce3d078d/0xf62499ab9278f46a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1db expref: 213 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.026482:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ce3d078d/0xf62499ab9278f46a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1db expref: 213 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.026491:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510720000:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.026513:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9aedcd80 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.026559:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a84e00/2, rc 0, transno 4295070666 00000100:00100000:0.0:1785220582.026579:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510716416:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.026592:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a86a00/2, rc 0, transno 4295070668 00000100:00100000:0.0:1785220582.026595:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a86a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510716032:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.026606:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725880/2, rc 0, transno 4295070093 00000100:00100000:1.0:1785220582.026722:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.026730:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510720128:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.026766:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.026786:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510720256:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.026787:0:14161:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510720128 00000100:00100000:1.0:1785220582.026790:0:14161:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510720128 00000100:00100000:2.0:1785220582.026814:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.026814:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284547 00000100:00100000:1.0:1785220582.026834:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+46:3654:x1871938510720128:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.026837:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510720256 00000100:00100000:2.0:1785220582.026839:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510720256 00002000:00080000:1.0:1785220582.026845:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29368 count 1 00010000:00010000:1.0:1785220582.026856:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.026863:0:14170:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284548 00000100:00100000:2.0:1785220582.026869:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+47:3652:x1871938510720256:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.026875:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007d8e2bc6/0xf62499ab92790a27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3ef expref: 184 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.026879:0:14170:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29369 count 1 00010000:00010000:1.0:1785220582.026882:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b8:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.026884:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.026888:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.026889:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.026900:0:14170:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ca850e5/0xf62499ab92790acf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3fd expref: 184 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.026906:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007d8e2bc6/0xf62499ab92790a27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3ef expref: 184 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.026907:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72b9:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.026910:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.026914:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.026916:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007d8e2bc6/0xf62499ab92790a27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3ef expref: 184 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.026926:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510720384:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.026929:0:14170:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ca850e5/0xf62499ab92790acf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3fd expref: 184 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.026938:0:14170:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ca850e5/0xf62499ab92790acf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3fd expref: 184 pid: 14183 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.026947:0:14170:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510720512:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.026952:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.026966:0:14170:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59240 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.026987:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a85180/2, rc 0, transno 4295070669 00000100:00100000:2.0:1785220582.027004:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510716160:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.027249:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510715648:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.027302:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.027309:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510720768:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.027335:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.027352:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510720768 00000100:00100000:3.0:1785220582.027356:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510720768 00000100:00100000:2.0:1785220582.027386:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220582.027388:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284549 00000100:00100000:2.0:1785220582.027393:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510720640:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.027395:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+48:3655:x1871938510720768:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.027407:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29370 count 1 00000100:00100000:2.0:1785220582.027417:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.027419:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.027430:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e6505c96/0xf62499ab92790b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f40b expref: 184 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.027437:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ba:0x0].0x0 since=20s v=(0 0) 00010000:00010000:3.0:1785220582.027439:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.027443:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.027464:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e6505c96/0xf62499ab92790b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f40b expref: 184 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.027471:0:14205:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510720640 00010000:00010000:3.0:1785220582.027474:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e6505c96/0xf62499ab92790b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f40b expref: 184 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.027474:0:14205:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510720640 00000100:00100000:3.0:1785220582.027484:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510720896:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.027508:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284550 00000100:00100000:3.0:1785220582.027514:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.027514:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1243:3652:x1871938510720640:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.027526:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29331 count 1 00010000:00010000:0.0:1785220582.027538:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.027551:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009d1eac96/0xf62499ab9278f50b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f0 expref: 213 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.027557:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7293:0x0].0x0 since=22s v=(0 0) 00010000:00010000:0.0:1785220582.027560:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7293:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.027564:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.027583:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009d1eac96/0xf62499ab9278f50b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f0 expref: 213 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.027593:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009d1eac96/0xf62499ab9278f50b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f0 expref: 213 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.027603:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510721024:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.027702:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a4d5600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.032868:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510717696:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.032936:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510717952:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.032962:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.032995:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510717568:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.033012:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.033044:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.033077:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.033140:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.033141:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.033160:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.033173:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519966976 00000100:00100000:2.0:1785220582.033175:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.033179:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519966976 00010000:00010000:1.0:1785220582.033213:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000abc71b06/0xf62499ab9278f344 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0f4 expref: 214 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.033224:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.033247:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519967232 00000100:00100000:1.0:1785220582.033251:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519967232 00000100:00100000:2.0:1785220582.033257:0:14170:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510720512:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.033262:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000051db8067/0xf62499ab9278f424 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b1 expref: 215 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.033269:0:14170:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.033279:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968000 00000100:00100000:1.0:1785220582.033281:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968000 00010000:00010000:1.0:1785220582.033292:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000053751f2b/0xf62499ab9278f3ad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f15d expref: 216 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.033315:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519967104 00000100:00100000:1.0:1785220582.033318:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519967104 00000100:00100000:2.0:1785220582.033319:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519967488 00000100:00100000:2.0:1785220582.033322:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519967488 00010000:00010000:1.0:1785220582.033329:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c679e409/0xf62499ab9278f383 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f133 expref: 217 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.033335:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d7558fe1/0xf62499ab92790821 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3c5 expref: 185 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.033345:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968256 00000100:00100000:1.0:1785220582.033347:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968256 00000100:00100000:2.0:1785220582.033354:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100468 00010000:00010000:1.0:1785220582.033358:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006dd11019/0xf62499ab927908c9 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3d3 expref: 186 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.033361:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+217:2406:x1871938519966976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.033375:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.033376:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100469 00010000:00010000:2.0:1785220582.033380:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000abc71b06/0xf62499ab9278f344 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0f4 expref: 217 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.033381:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+217:2406:x1871938519967232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.033388:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.033392:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000051db8067/0xf62499ab9278f424 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b1 expref: 217 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.033398:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000abc71b06/0xf62499ab9278f344 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0f4 expref: 217 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033406:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000051db8067/0xf62499ab9278f424 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b1 expref: 217 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.033423:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abc71b06/0xf62499ab9278f344 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f0f4 expref: 217 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033423:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000051db8067/0xf62499ab9278f424 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1b1 expref: 217 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.033439:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.033439:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.033444:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.033444:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.033450:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033450:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.033461:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033461:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033481:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.033482:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.033552:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510720896:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.033568:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.033583:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.033590:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510719104:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.033591:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.033601:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.033603:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.033611:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.033626:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.033697:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510719488:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.033711:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.033733:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510719872:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.033743:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.033776:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510721024:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.033787:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.033803:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+215:2406:x1871938519967232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 420us (724us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.033806:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100470 00000100:00100000:1.0:1785220582.033810:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100469 00000100:00100000:0.0:1785220582.033813:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+215:2406:x1871938519968000:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.033821:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.033827:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000053751f2b/0xf62499ab9278f3ad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f15d expref: 215 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.033837:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000053751f2b/0xf62499ab9278f3ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f15d expref: 214 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.033840:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100471 00000100:00100000:1.0:1785220582.033847:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a84380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+214:2406:x1871938519967104:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.033851:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000053751f2b/0xf62499ab9278f3ad lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f15d expref: 214 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033854:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.033858:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c679e409/0xf62499ab9278f383 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f133 expref: 214 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.033861:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.033864:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.033869:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033874:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c679e409/0xf62499ab9278f383 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f133 expref: 213 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.033877:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033885:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c679e409/0xf62499ab9278f383 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f133 expref: 213 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.033893:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.033895:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.033897:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x728d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.033901:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033908:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033928:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.033935:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+212:2406:x1871938519966976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 574us (891us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.033943:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100468 00000100:00100000:1.0:1785220582.033944:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a84380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+212:2406:x1871938519967104:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (783us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.033949:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100471 00000100:00100000:2.0:1785220582.033956:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100472 00000100:00100000:1.0:1785220582.033960:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100473 00000100:00100000:2.0:1785220582.033962:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e07480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+186:2406:x1871938519967488:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.033965:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94e07b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+186:2406:x1871938519968256:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.033971:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.033971:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.033977:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d7558fe1/0xf62499ab92790821 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3c5 expref: 186 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033977:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006dd11019/0xf62499ab927908c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3d3 expref: 186 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.033995:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006dd11019/0xf62499ab927908c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3d3 expref: 186 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.033996:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d7558fe1/0xf62499ab92790821 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3c5 expref: 186 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034009:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.034009:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006dd11019/0xf62499ab927908c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3d3 expref: 186 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034012:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.034016:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034018:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.034021:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.034024:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034025:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034033:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034035:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.034046:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.034054:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e07480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+185:2406:x1871938519967488:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (877us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.034059:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94e07b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+185:2406:x1871938519968256:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (835us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.034060:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100472 00000100:00100000:1.0:1785220582.034065:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100473 00010000:00010000:2.0:1785220582.034081:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034121:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.034121:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510719360:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.034130:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034134:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034137:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034150:0:14175:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034151:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.034162:0:14175:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034169:0:14175:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034176:0:14175:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034189:0:14175:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220582.034244:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1246:3652:x1871938510717440:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15309us (15922us total) trans 4295070095 rc 0/0 00000100:00100000:2.0:1785220582.034253:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284537 00000100:00100000:1.0:1785220582.034259:0:14175:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1246:3654:x1871938510716928:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19052us (19493us total) trans 4295070096 rc 0/0 00000100:00100000:1.0:1785220582.034265:0:14175:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284535 00010000:00010000:2.0:1785220582.034271:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.034277:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034285:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.034289:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.034289:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510720384:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.034299:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034300:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034312:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034316:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034320:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a533c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034323:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034330:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034338:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.034338:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034342:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034351:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.034353:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.034356:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034362:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034371:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034378:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5335 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034390:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.034393:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034399:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97734400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.034402:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034408:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034417:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034424:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000653106ba/0xf62499ab927a5327 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.034430:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+49:3654:x1871938510718464:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8162us (8352us total) trans 4295070670 rc 0/0 00000100:00100000:1.0:1785220582.034436:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284545 00010000:00010000:2.0:1785220582.034437:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.034441:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034447:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.034450:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034457:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.034461:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510719616:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.034466:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034472:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034474:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a536d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034488:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.034489:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034490:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.034491:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.034498:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034506:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034513:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.034523:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a84a80/2, rc 0, transno 4295070095 00010000:00010000:1.0:1785220582.034532:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.034534:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510717440:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.034541:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.034545:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034551:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97735000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.034555:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034562:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034572:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034579:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000096fcc85/0xf62499ab927a532e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x728d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034592:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.034593:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.034595:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.034602:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9573fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1245:3652:x1871938510717312:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15683us (16515us total) trans 4295070097 rc 0/0 00000004:00080000:2.0:1785220582.034607:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a86680/2, rc 0, transno 4295070097 00000100:00100000:1.0:1785220582.034609:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284536 00000100:00100000:2.0:1785220582.034611:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a86680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510717312:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.034742:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.034774:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519967360 00000100:00100000:2.0:1785220582.034779:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519967360 00010000:00010000:2.0:1785220582.034799:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002456099a/0xf62499ab9278f455 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1cd expref: 211 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.034815:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.034828:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968512 00000100:00100000:2.0:1785220582.034831:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968512 00010000:00010000:2.0:1785220582.034843:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007d8e2bc6/0xf62499ab92790a27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3ef expref: 184 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.034850:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.034859:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519967744 00000100:00100000:2.0:1785220582.034862:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519967744 00000100:00100000:1.0:1785220582.034869:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100474 00010000:00010000:2.0:1785220582.034872:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000080047ebe/0xf62499ab9278f447 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1bf expref: 212 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.034877:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+212:2404:x1871938519967360:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.034886:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220582.034891:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100475 00010000:00010000:1.0:1785220582.034892:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002456099a/0xf62499ab9278f455 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1cd expref: 212 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.034897:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+184:2406:x1871938519968512:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.034902:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002456099a/0xf62499ab9278f455 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1cd expref: 212 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034903:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.034907:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007d8e2bc6/0xf62499ab92790a27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3ef expref: 184 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034917:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007d8e2bc6/0xf62499ab92790a27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3ef expref: 184 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034917:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002456099a/0xf62499ab9278f455 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1cd expref: 212 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034928:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7291:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.034929:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d8e2bc6/0xf62499ab92790a27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3ef expref: 184 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034932:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7291:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.034937:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034938:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.034941:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.034946:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.034946:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034955:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.034970:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.034970:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.035048:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+183:2406:x1871938519968512:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 150us (230us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.035054:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100475 00000100:00100000:2.0:1785220582.035069:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100476 00000100:00100000:1.0:1785220582.035069:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+211:2404:x1871938519967360:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 193us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.035075:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+211:2404:x1871938519967744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.035076:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100474 00010000:00010000:2.0:1785220582.035082:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.035113:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035118:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000080047ebe/0xf62499ab9278f447 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1bf expref: 210 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.035123:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035128:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000080047ebe/0xf62499ab9278f447 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1bf expref: 210 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.035132:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.035139:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035142:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000080047ebe/0xf62499ab9278f447 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1bf expref: 210 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035151:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7290:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.035153:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.035154:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7290:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.035158:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035170:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035186:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.035204:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+209:2404:x1871938519967744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 128us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.035210:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100476 00010000:00010000:2.0:1785220582.035229:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.035233:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1245:3653:x1871938510718848:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10057us (10224us total) trans 4295070098 rc 0/0 00010000:00010000:2.0:1785220582.035238:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.035240:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284540 00010000:00010000:2.0:1785220582.035246:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035253:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035261:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1785220582.035269:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb66680/2, rc 0, transno 4295070670 00000100:00100000:1.0:1785220582.035279:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510718464:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.035339:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+49:3654:x1871938510720128:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8518us (8571us total) trans 4295070671 rc 0/0 00000100:00100000:2.0:1785220582.035346:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284547 00010000:00010000:2.0:1785220582.035362:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.035367:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035375:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.035379:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035386:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035396:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035403:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a534a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7291:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035416:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.035419:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035425:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.035429:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035436:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035445:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035453:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a537b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.035465:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.035467:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.035469:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.035664:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ee300/2, rc 0, transno 4295070096 00000100:00100000:1.0:1785220582.035670:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510716928:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.035678:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.035690:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510721152:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.035794:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.035908:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ef800/2, rc 0, transno 4295070671 00000100:00100000:1.0:1785220582.035917:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510720128:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.035986:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510721152 00000100:00100000:2.0:1785220582.035991:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510721152 00000100:00100000:2.0:1785220582.036030:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284551 00000100:00100000:2.0:1785220582.036038:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+49:3654:x1871938510721152:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.036061:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29371 count 1 00010000:00010000:2.0:1785220582.036082:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.036265:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000067356c5e/0xf62499ab92790c3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f419 expref: 182 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.036273:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72bb:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.036277:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.036280:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.036307:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000067356c5e/0xf62499ab92790c3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f419 expref: 182 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.036319:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000067356c5e/0xf62499ab92790c3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f419 expref: 182 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.036333:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510721408:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.036379:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.036396:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510721280:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.036422:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.036529:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510721280 00000100:00100000:1.0:1785220582.036533:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510721280 00000100:00100000:1.0:1785220582.036568:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284552 00000100:00100000:1.0:1785220582.036575:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+50:3652:x1871938510721280:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.036589:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29372 count 1 00010000:00010000:1.0:1785220582.036604:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.036617:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c2b95c3a/0xf62499ab92790cf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f427 expref: 182 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.036624:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72bc:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.036627:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.036630:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.036649:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c2b95c3a/0xf62499ab92790cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f427 expref: 182 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.036658:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c2b95c3a/0xf62499ab92790cf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f427 expref: 182 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.036675:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510721536:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.036700:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba0c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.037311:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.037342:0:14195:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7558fe1/0xf62499ab92790821 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3c5 expref: 182 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.037363:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510719232:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.037383:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037392:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037399:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037408:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037417:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037436:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.037553:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+51:3653:x1871938510718336:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12406us (12580us total) trans 4295070672 rc 0/0 00000100:00100000:1.0:1785220582.037560:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284539 00000100:00100000:1.0:1785220582.037592:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510720000:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.037606:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.037646:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510719744:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.037657:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037671:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.037677:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037684:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.037688:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.037694:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.037697:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037714:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037726:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000812fbe23/0xf62499ab927a5343 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.037742:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.037743:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.037745:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.040140:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.040174:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968768 00000100:00100000:1.0:1785220582.040182:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968768 00000100:00100000:2.0:1785220582.040190:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.040212:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.040217:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e6505c96/0xf62499ab92790b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f40b expref: 182 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.040227:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968128 00000100:00100000:2.0:1785220582.040230:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968128 00010000:00010000:2.0:1785220582.040247:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000592ea131/0xf62499ab927909a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e1 expref: 183 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.040247:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519967616 00000100:00100000:1.0:1785220582.040249:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519967616 00010000:00010000:1.0:1785220582.040260:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fde976c3/0xf62499ab9279079c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b7 expref: 184 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.040268:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100477 00000100:00100000:2.0:1785220582.040276:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e82a2a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+184:2406:x1871938519968768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.040291:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.040297:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e6505c96/0xf62499ab92790b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f40b expref: 184 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.040308:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e6505c96/0xf62499ab92790b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f40b expref: 184 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.040328:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e6505c96/0xf62499ab92790b77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f40b expref: 184 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.040350:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.040355:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.040360:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.040370:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.040396:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.040440:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e82a2a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+183:2406:x1871938519968768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 164us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.040490:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94acb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1244:3655:x1871938510717824:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19501us (19922us total) trans 4295070094 rc 0/0 00000100:00100000:0.0:1785220582.040496:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+208:2406:x1871938519968000:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 6683us (7355us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.040499:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284538 00010000:00010000:3.0:1785220582.040532:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040542:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040552:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040560:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040575:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.040709:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+51:3655:x1871938510720768:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13315us (13374us total) trans 4295070673 rc 0/0 00000100:00100000:3.0:1785220582.040716:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284549 00010000:00010000:3.0:1785220582.040738:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.040743:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040751:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.040766:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040782:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040794:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040802:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a5389 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.040819:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.040821:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.040822:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.040914:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb65880/2, rc 0, transno 4295070672 00000100:00100000:3.0:1785220582.040925:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718336:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.040982:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601ca80/2, rc 0, transno 4295070098 00000100:00100000:3.0:1785220582.040987:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718848:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.041001:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510721664:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.041029:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.041232:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100478 00000100:00100000:1.0:1785220582.041242:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+183:2404:x1871938519968128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.041251:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.041256:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000592ea131/0xf62499ab927909a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e1 expref: 183 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041266:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000592ea131/0xf62499ab927909a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e1 expref: 183 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041280:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000592ea131/0xf62499ab927909a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e1 expref: 183 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041297:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.041301:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.041305:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041313:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041329:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.041348:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+182:2404:x1871938519968128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (1156us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.041354:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100478 00000100:00100000:1.0:1785220582.041367:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100479 00000100:00100000:1.0:1785220582.041372:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99918000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+181:2404:x1871938519967616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.041378:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.041383:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fde976c3/0xf62499ab9279079c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b7 expref: 181 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041393:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fde976c3/0xf62499ab9279079c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b7 expref: 181 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041405:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fde976c3/0xf62499ab9279079c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b7 expref: 181 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041419:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.041422:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.041426:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041434:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041448:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.041462:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+180:2404:x1871938519967616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (1249us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.041467:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100479 00010000:00010000:1.0:1785220582.041488:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041496:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041505:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041512:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041528:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.041611:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+51:3653:x1871938510718208:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16280us (16561us total) trans 4295070674 rc 0/0 00000100:00100000:1.0:1785220582.041618:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284541 00000100:00100000:1.0:1785220582.041630:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510721664 00000100:00100000:1.0:1785220582.041634:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510721664 00000100:00100000:1.0:1785220582.041658:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284553 00000100:00100000:1.0:1785220582.041663:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+51:3653:x1871938510721664:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.041682:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29373 count 1 00010000:00010000:1.0:1785220582.041698:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041712:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bffa3923/0xf62499ab92790da0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f435 expref: 179 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041719:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72bd:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.041722:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.041726:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041745:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bffa3923/0xf62499ab92790da0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f435 expref: 179 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.041769:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bffa3923/0xf62499ab92790da0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f435 expref: 179 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.041781:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510721792:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.041818:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.041923:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.041929:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.041938:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.041942:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.041951:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.041962:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.041969:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5351 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.041984:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.041986:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.041988:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.042008:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb67100/2, rc 0, transno 4295070674 00000100:00100000:3.0:1785220582.042016:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718208:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.042316:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.042378:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.042409:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.042436:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.042488:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100477 00000100:00100000:2.0:1785220582.042533:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968640 00000100:00100000:2.0:1785220582.042537:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968640 00010000:00010000:2.0:1785220582.042554:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ca850e5/0xf62499ab92790acf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3fd expref: 179 pid: 14183 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.042576:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519967872 00000100:00100000:2.0:1785220582.042579:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519967872 00010000:00010000:2.0:1785220582.042591:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000054ba2e46/0xf62499ab927906e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a9 expref: 180 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.042607:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968896 00000100:00100000:2.0:1785220582.042621:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968896 00010000:00010000:2.0:1785220582.042637:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009d1eac96/0xf62499ab9278f50b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f0 expref: 209 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.042662:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519968384 00000100:00100000:2.0:1785220582.042665:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519968384 00010000:00010000:2.0:1785220582.042677:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ce3d078d/0xf62499ab9278f46a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1db expref: 210 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.042697:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100480 00000100:00100000:2.0:1785220582.042704:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+180:2406:x1871938519968640:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.042713:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.042717:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ca850e5/0xf62499ab92790acf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3fd expref: 180 pid: 14183 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042728:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ca850e5/0xf62499ab92790acf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3fd expref: 180 pid: 14183 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042743:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ca850e5/0xf62499ab92790acf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3fd expref: 180 pid: 14183 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042763:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.042767:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.042772:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042780:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042804:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.042830:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+179:2406:x1871938519968640:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 127us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.042837:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100480 00000100:00100000:2.0:1785220582.042852:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100481 00000100:00100000:2.0:1785220582.042858:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+178:2404:x1871938519967872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.042865:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.042870:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000054ba2e46/0xf62499ab927906e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a9 expref: 178 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042880:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000054ba2e46/0xf62499ab927906e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a9 expref: 178 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042893:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054ba2e46/0xf62499ab927906e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a9 expref: 178 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042903:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.042906:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.042910:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042918:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.042941:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.042959:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+177:2404:x1871938519967872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (581us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.042966:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100481 00000100:00100000:2.0:1785220582.042980:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100482 00000100:00100000:2.0:1785220582.042986:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+210:2406:x1871938519968896:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.042994:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.042999:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009d1eac96/0xf62499ab9278f50b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f0 expref: 210 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043009:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009d1eac96/0xf62499ab9278f50b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f0 expref: 210 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043021:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009d1eac96/0xf62499ab9278f50b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1f0 expref: 210 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043031:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7293:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043034:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7293:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043038:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043044:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043071:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.043107:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+209:2406:x1871938519968896:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 122us (698us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.043114:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100482 00000100:00100000:2.0:1785220582.043129:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100483 00000100:00100000:2.0:1785220582.043136:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+208:2404:x1871938519968384:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.043143:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.043148:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ce3d078d/0xf62499ab9278f46a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1db expref: 208 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043158:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ce3d078d/0xf62499ab9278f46a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1db expref: 208 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043170:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ce3d078d/0xf62499ab9278f46a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1db expref: 208 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043181:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7292:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043184:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7292:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043189:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043196:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043218:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.043236:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+207:2404:x1871938519968384:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (799us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.043242:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100483 00000100:00100000:2.0:1785220582.043285:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510721408:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.043303:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043324:0:14170:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043332:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043341:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043348:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043363:0:14170:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.043419:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.043449:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.043500:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+52:3652:x1871938510720256:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16629us (16684us total) trans 4295070675 rc 0/0 00000100:00100000:2.0:1785220582.043507:0:14170:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284548 00000100:00100000:2.0:1785220582.043527:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969024 00000100:00100000:2.0:1785220582.043531:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969024 00010000:00010000:2.0:1785220582.043546:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000067356c5e/0xf62499ab92790c3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f419 expref: 177 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.043565:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969152 00000100:00100000:2.0:1785220582.043568:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969152 00010000:00010000:2.0:1785220582.043581:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c2b95c3a/0xf62499ab92790cf1 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f427 expref: 178 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.043598:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100484 00000100:00100000:2.0:1785220582.043605:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90701450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+178:2406:x1871938519969024:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.043613:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.043618:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000067356c5e/0xf62499ab92790c3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f419 expref: 178 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043627:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000067356c5e/0xf62499ab92790c3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f419 expref: 178 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043640:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067356c5e/0xf62499ab92790c3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f419 expref: 178 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043651:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043654:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043658:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043672:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043689:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.043708:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90701450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+177:2406:x1871938519969024:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (289us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.043715:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100484 00000100:00100000:2.0:1785220582.043747:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100485 00000100:00100000:2.0:1785220582.043764:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+176:2406:x1871938519969152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.043772:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.043777:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c2b95c3a/0xf62499ab92790cf1 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f427 expref: 176 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043787:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c2b95c3a/0xf62499ab92790cf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f427 expref: 176 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043800:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043803:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.043806:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043813:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.043816:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f726300/2, rc 0, transno 4295070675 00010000:00010000:2.0:1785220582.043830:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.043847:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+176:2406:x1871938519969152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.043853:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100485 00010000:00010000:2.0:1785220582.043875:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043883:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043891:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043898:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.043911:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.043919:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.043926:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.043934:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.043938:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.043947:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.043960:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.043967:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5382 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.043984:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+52:3654:x1871938510721152:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7947us (8189us total) trans 4295070676 rc 0/0 00010000:00010000:1.0:1785220582.043985:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.043989:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.043990:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284551 00010000:00010000:1.0:1785220582.043995:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.043998:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.044006:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044016:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.044016:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.044024:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5397 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044025:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.044037:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044038:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.044038:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.044040:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044046:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044058:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.044060:0:14179:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2b95c3a/0xf62499ab92790cf1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f427 expref: 175 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.044070:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0000_UUID:14179:1871938510721536:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.044082:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.044145:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+52:3654:x1871938510718080:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18216us (19368us total) trans 4295070677 rc 0/0 00000100:00100000:2.0:1785220582.044150:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284543 00010000:00010000:2.0:1785220582.044168:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.044173:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044180:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.044183:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044190:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044202:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044209:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a535f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044225:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044228:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044229:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044239:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044249:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044257:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044264:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044275:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.044277:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510720256:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.044358:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+1244:3654:x1871938510718976:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17951us (18214us total) trans 4295070099 rc 0/0 00000100:00100000:2.0:1785220582.044365:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284546 00010000:00010000:2.0:1785220582.044385:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.044390:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044397:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ada00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.044400:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044409:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044420:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044427:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0447b82/0xf62499ab927a5374 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7292:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.044443:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044444:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044446:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.044455:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.045266:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.045279:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.045288:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.045295:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.045322:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.045377:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.045383:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045396:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.045399:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045406:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045419:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.045420:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+52:3655:x1871938510718592:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19359us (19414us total) trans 4295070678 rc 0/0 00010000:00010000:3.0:1785220582.045426:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a5366 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.045427:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284544 00010000:00010000:3.0:1785220582.045441:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.045443:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.045445:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.045499:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510722176:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.045518:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510722304:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.045533:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.045558:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510722176 00000100:00100000:2.0:1785220582.045563:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510722176 00000100:00100000:2.0:1785220582.045600:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284554 00000100:00100000:2.0:1785220582.045608:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1244:3652:x1871938510722176:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.045644:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.045651:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.045660:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.045672:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.045724:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.045835:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99918380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+52:3652:x1871938510721280:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9259us (9411us total) trans 4295070679 rc 0/0 00000100:00100000:1.0:1785220582.045843:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284552 00010000:00010000:3.0:1785220582.045930:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.045936:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045942:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.045945:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.045956:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29332 count 1 00010000:00010000:3.0:1785220582.045957:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045967:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045973:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a539e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.045976:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045989:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.045990:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000771f932e/0xf62499ab9278f520 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1fe expref: 206 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.045991:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.045993:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.045999:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7294:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.046003:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7294:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.046008:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.046016:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725c00/2, rc 0, transno 4295070679 00000100:00100000:3.0:1785220582.046025:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510721280:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.046031:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000771f932e/0xf62499ab9278f520 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1fe expref: 206 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.046042:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000771f932e/0xf62499ab9278f520 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1fe expref: 206 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.046054:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510722688:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.046055:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510722816:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.046115:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.046119:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.046132:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510722816 00000100:00100000:2.0:1785220582.046135:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510722816 00000100:00100000:2.0:1785220582.046158:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284555 00000100:00100000:2.0:1785220582.046164:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+52:3652:x1871938510722816:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.046176:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29376 count 1 00010000:00010000:2.0:1785220582.046187:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.046198:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e5032e17/0xf62499ab92790f59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f45f expref: 174 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.046205:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c0:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.046208:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.046211:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.046231:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e5032e17/0xf62499ab92790f59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f45f expref: 174 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.046240:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e5032e17/0xf62499ab92790f59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f45f expref: 174 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.046258:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510722944:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.046275:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.046281:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.046293:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.046304:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510721920:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.046329:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.046381:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510722432:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.046401:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.046500:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724380/2, rc 0, transno 4295070676 00000100:00100000:1.0:1785220582.046510:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510721152:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.046609:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100470 00010000:00010000:0.0:1785220582.046663:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.046672:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.046679:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.046685:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.046700:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.046812:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1245:3652:x1871938510720640:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19298us (19393us total) trans 4295070100 rc 0/0 00000100:00100000:0.0:1785220582.046820:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284550 00000100:00100000:1.0:1785220582.046990:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.047000:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f726680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510723072:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.047026:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.047050:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510722304 00000100:00100000:3.0:1785220582.047055:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510722304 00000100:00100000:2.0:1785220582.047419:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510722432 00000100:00100000:2.0:1785220582.047424:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510722432 00000100:00100000:3.0:1785220582.047432:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510721920 00000100:00100000:3.0:1785220582.047435:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510721920 00000100:00100000:2.0:1785220582.047451:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284556 00000100:00100000:3.0:1785220582.047455:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284557 00000100:00100000:2.0:1785220582.047458:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1246:3653:x1871938510722304:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.047460:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1246:3654:x1871938510722432:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.047473:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29334 count 1 00002000:00080000:2.0:1785220582.047473:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29333 count 1 00010000:00010000:3.0:1785220582.047489:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.047489:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.047504:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000474126e2/0xf62499ab9278f52e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f20c expref: 205 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.047505:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002c3214af/0xf62499ab9278f5dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f21a expref: 205 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.047513:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7296:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.047513:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7295:0x0].0x0 since=21s v=(0 0) 00010000:00010000:3.0:1785220582.047517:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7296:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.047517:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7295:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.047522:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.047522:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.047544:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002c3214af/0xf62499ab9278f5dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f21a expref: 205 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.047544:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000474126e2/0xf62499ab9278f52e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f20c expref: 205 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.047557:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002c3214af/0xf62499ab9278f5dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f21a expref: 205 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.047557:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000474126e2/0xf62499ab9278f52e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f20c expref: 205 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.047568:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510723328:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.047571:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0001_UUID:14232:1871938510723200:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.047730:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284558 00000100:00100000:3.0:1785220582.047732:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.047738:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+53:3654:x1871938510721920:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.047751:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29374 count 1 00010000:00010000:0.0:1785220582.047774:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.047785:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000094b9dd89/0xf62499ab92790e33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f443 expref: 174 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.047792:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72be:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.047795:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.047798:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.047816:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000094b9dd89/0xf62499ab92790e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f443 expref: 174 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.047826:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000094b9dd89/0xf62499ab92790e33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f443 expref: 174 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.047837:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0000_UUID:14205:1871938510723456:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.047871:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59d80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.047969:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.047985:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510723072 00000100:00100000:0.0:1785220582.047988:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510723072 00000100:00100000:3.0:1785220582.047989:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.048013:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284559 00000100:00100000:0.0:1785220582.048019:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+54:3652:x1871938510723072:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.048031:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29377 count 1 00010000:00010000:0.0:1785220582.048043:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.048054:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ce3fd24/0xf62499ab9279101d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f46d expref: 174 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.048060:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c1:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.048063:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.048066:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.048129:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969280 00000100:00100000:2.0:1785220582.048145:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969280 00010000:00010000:2.0:1785220582.048165:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bffa3923/0xf62499ab92790da0 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f435 expref: 175 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.048189:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100486 00000100:00100000:2.0:1785220582.048196:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+175:2407:x1871938519969280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.048204:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.048209:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bffa3923/0xf62499ab92790da0 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f435 expref: 175 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.048220:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bffa3923/0xf62499ab92790da0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f435 expref: 175 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.048238:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.048241:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.048246:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.048254:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.048274:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.048295:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+175:2407:x1871938519969280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.048302:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100486 00010000:00010000:0.0:1785220582.048451:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ce3fd24/0xf62499ab9279101d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f46d expref: 174 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.048464:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ce3fd24/0xf62499ab9279101d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f46d expref: 174 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.048474:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510723584:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.048501:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.048525:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.048533:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.048542:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.048551:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.049914:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.050098:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1247:3653:x1871938510718720:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24249us (25000us total) trans 4295070101 rc 0/0 00000100:00100000:0.0:1785220582.050107:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284542 00010000:00010000:2.0:1785220582.050416:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.050425:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050436:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.050440:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050451:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050470:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050478:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5390 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7293:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050502:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.050506:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050518:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.050521:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050528:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050539:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050545:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5358 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7290:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.050560:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.050562:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.050564:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.050600:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.050704:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510722048:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.050741:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.050816:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510722560:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.050838:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.050904:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510722048 00000100:00100000:2.0:1785220582.050911:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510722048 00000100:00100000:2.0:1785220582.050947:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510722560 00000100:00100000:2.0:1785220582.050949:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510722560 00010000:00010000:1.0:1785220582.050964:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bffa3923/0xf62499ab92790da0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f435 expref: 174 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.050967:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284560 00000100:00100000:2.0:1785220582.050974:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+55:3655:x1871938510722048:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.050977:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510721792:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.050995:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.050998:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29375 count 1 00010000:00010000:1.0:1785220582.051008:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051015:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051016:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051024:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051031:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051035:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ca27c79e/0xf62499ab92790ec6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f451 expref: 173 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051043:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72bf:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.051046:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.051048:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.051052:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051077:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ca27c79e/0xf62499ab92790ec6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f451 expref: 173 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051115:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ca27c79e/0xf62499ab92790ec6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f451 expref: 173 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.051128:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0000_UUID:14172:1871938510723712:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.051238:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284561 00000100:00100000:0.0:1785220582.051246:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99be5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1247:3655:x1871938510722560:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.051258:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29335 count 1 00010000:00010000:0.0:1785220582.051272:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.051283:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000111af165/0xf62499ab9278f638 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f228 expref: 205 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.051289:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7297:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220582.051291:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7297:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.051294:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.051313:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000111af165/0xf62499ab9278f638 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f228 expref: 205 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.051317:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+56:3653:x1871938510721664:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9653us (10287us total) trans 4295070680 rc 0/0 00000100:00100000:1.0:1785220582.051324:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284553 00010000:00010000:0.0:1785220582.051324:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000111af165/0xf62499ab9278f638 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f228 expref: 205 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.051334:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510723840:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.051345:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.051349:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051357:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.051360:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051368:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051380:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051386:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a53a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.051387:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd596c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.051403:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.051405:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.051406:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.051432:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be4a80/2, rc 0, transno 4295070100 00000100:00100000:1.0:1785220582.051451:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510720640:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.051491:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c0700/2, rc 0, transno 4295070680 00000100:00100000:1.0:1785220582.051500:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510721664:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.051532:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601fb80/2, rc 0, transno 4295070101 00000100:00100000:1.0:1785220582.051547:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510718720:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.051564:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601df80/2, rc 0, transno 4295070099 00000100:00100000:1.0:1785220582.051567:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510718976:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.051575:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94acad80/2, rc 0, transno 4295070677 00000100:00100000:1.0:1785220582.051580:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510718080:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.051617:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510724352:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.051641:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.051662:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.051668:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510724224:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.051688:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.051711:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510724352 00000100:00100000:2.0:1785220582.051714:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510724352 00000100:00100000:1.0:1785220582.051718:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99919c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510723968:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.051737:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510724224 00000100:00100000:1.0:1785220582.051738:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.051740:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510724224 00000100:00100000:2.0:1785220582.051766:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510723968 00000100:00100000:1.0:1785220582.051768:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510724480:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.051769:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510723968 00000100:00100000:2.0:1785220582.051792:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284562 00000100:00100000:1.0:1785220582.051792:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.051798:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+56:3652:x1871938510724352:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.051810:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29379 count 1 00000100:00100000:1.0:1785220582.051810:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510724480 00000100:00100000:1.0:1785220582.051812:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510724480 00010000:00010000:2.0:1785220582.051822:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051835:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000070389ebc/0xf62499ab92791197 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f489 expref: 173 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051841:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c3:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.051845:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c3:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.051846:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284563 00010000:00010000:2.0:1785220582.051849:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.051853:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1250:3652:x1871938510724224:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.051863:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29337 count 1 00010000:00010000:2.0:1785220582.051866:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000070389ebc/0xf62499ab92791197 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f489 expref: 173 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051874:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051875:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000070389ebc/0xf62499ab92791197 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f489 expref: 173 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.051885:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c5255cb3/0xf62499ab9278f734 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f244 expref: 205 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.051886:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510724608:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.051893:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7299:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.051896:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7299:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.051899:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.051909:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a4d5600 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.051915:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c5255cb3/0xf62499ab9278f734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f244 expref: 205 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.051925:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284564 00010000:00010000:1.0:1785220582.051925:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c5255cb3/0xf62499ab9278f734 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f244 expref: 205 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.051932:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1250:3654:x1871938510723968:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.051941:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510724736:192.168.201.51@tcp:104: 00002000:00080000:2.0:1785220582.051943:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29336 count 1 00010000:00010000:2.0:1785220582.051953:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.051962:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.051964:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002d48748d/0xf62499ab9278f6e0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f236 expref: 205 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.051971:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7298:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.051974:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7298:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.051976:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284565 00010000:00010000:2.0:1785220582.051978:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.051982:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d86ecb850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1250:3654:x1871938510724480:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.051993:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29338 count 1 00010000:00010000:2.0:1785220582.051998:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002d48748d/0xf62499ab9278f6e0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f236 expref: 205 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052004:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052007:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002d48748d/0xf62499ab9278f6e0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f236 expref: 205 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052014:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bb013b86/0xf62499ab9278f7dc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f252 expref: 205 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052017:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510724864:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.052022:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x729a:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.052024:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.052028:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052039:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec20c8540 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.052044:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bb013b86/0xf62499ab9278f7dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f252 expref: 205 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052053:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bb013b86/0xf62499ab9278f7dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f252 expref: 205 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.052062:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510724992:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.052254:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510722688:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.052271:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052318:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510722944:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.052332:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052396:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0001_UUID:14232:1871938510723200:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.052409:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.052431:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba300 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.052448:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.052452:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.052454:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510724096:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.052469:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969536 00000100:00100000:2.0:1785220582.052472:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969536 00000100:00100000:1.0:1785220582.052480:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.052491:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e5032e17/0xf62499ab92790f59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f45f expref: 174 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.052500:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510724096 00000100:00100000:1.0:1785220582.052503:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510724096 00000100:00100000:2.0:1785220582.052514:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100487 00000100:00100000:2.0:1785220582.052520:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+174:2406:x1871938519969536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.052522:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284566 00000100:00100000:1.0:1785220582.052527:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+57:3654:x1871938510724096:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.052530:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.052535:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e5032e17/0xf62499ab92790f59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f45f expref: 174 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.052538:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29378 count 1 00010000:00010000:2.0:1785220582.052545:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e5032e17/0xf62499ab92790f59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f45f expref: 174 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052551:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052561:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5032e17/0xf62499ab92790f59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f45f expref: 174 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052561:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e24c0eee/0xf62499ab927910ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f47b expref: 174 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052569:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c2:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.052571:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.052572:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.052575:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.052576:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052580:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052594:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052594:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e24c0eee/0xf62499ab927910ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f47b expref: 173 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052604:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e24c0eee/0xf62499ab927910ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f47b expref: 173 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052611:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.052613:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510725120:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.052642:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bacc0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.052678:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.052712:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.052728:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969792 00000100:00100000:1.0:1785220582.052731:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969792 00000100:00100000:2.0:1785220582.052740:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+173:2406:x1871938519969536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 220us (287us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.052746:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000474126e2/0xf62499ab9278f52e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f20c expref: 206 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052747:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100487 00000100:00100000:2.0:1785220582.052772:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969408 00000100:00100000:2.0:1785220582.052775:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969408 00000100:00100000:1.0:1785220582.052777:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100488 00000100:00100000:1.0:1785220582.052782:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+207:2406:x1871938519969792:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.052786:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000771f932e/0xf62499ab9278f520 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1fe expref: 207 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052791:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.052795:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000474126e2/0xf62499ab9278f52e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f20c expref: 207 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052805:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100489 00010000:00010000:1.0:1785220582.052805:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000474126e2/0xf62499ab9278f52e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f20c expref: 207 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052810:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb66300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+207:2406:x1871938519969408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.052816:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.052820:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000474126e2/0xf62499ab9278f52e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f20c expref: 207 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052821:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000771f932e/0xf62499ab9278f520 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1fe expref: 207 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052830:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000771f932e/0xf62499ab9278f520 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1fe expref: 206 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052831:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7295:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.052834:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7295:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.052838:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052843:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000771f932e/0xf62499ab9278f520 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f1fe expref: 206 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.052847:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052858:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7294:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.052860:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7294:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.052863:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.052864:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052871:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.052876:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+205:2406:x1871938519969792:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (199us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.052883:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100488 00010000:00010000:2.0:1785220582.052885:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.052897:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb66300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+204:2406:x1871938519969408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 87us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.052903:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100489 00000100:00100000:1.0:1785220582.052916:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.052932:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969664 00000100:00100000:2.0:1785220582.052936:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969664 00010000:00010000:2.0:1785220582.052950:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002c3214af/0xf62499ab9278f5dd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f21a expref: 204 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.052970:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100490 00000100:00100000:2.0:1785220582.052976:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb67800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+204:2406:x1871938519969664:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.052983:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.052987:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002c3214af/0xf62499ab9278f5dd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f21a expref: 204 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.052996:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002c3214af/0xf62499ab9278f5dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f21a expref: 204 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053010:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7296:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.053013:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7296:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.053017:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053025:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.053027:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.053036:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.053049:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb67800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+204:2406:x1871938519969664:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 73us (133us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.053051:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.053055:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100490 00000100:00100000:2.0:1785220582.053065:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519969920 00000100:00100000:2.0:1785220582.053068:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519969920 00010000:00010000:2.0:1785220582.053080:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000094b9dd89/0xf62499ab92790e33 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f443 expref: 173 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.053232:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970048 00000100:00100000:2.0:1785220582.053236:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970048 00010000:00010000:2.0:1785220582.053251:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ce3fd24/0xf62499ab9279101d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f46d expref: 174 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.053345:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464780 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.053353:0:14194:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c3214af/0xf62499ab9278f5dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f21a expref: 203 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.053366:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510723328:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.053366:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510723584:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.053381:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.053381:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.053396:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.053401:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0000_UUID:14205:1871938510723456:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.053403:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.053410:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.053413:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.053421:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.053434:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f726a00/2, rc 0, transno 4295070673 00010000:00010000:3.0:1785220582.053435:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.053444:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510720768:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.053486:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601ed80/2, rc 0, transno 4295070678 00000100:00100000:0.0:1785220582.053492:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510718592:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.053523:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94acb100/2, rc 0, transno 4295070094 00000100:00100000:0.0:1785220582.053532:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510717824:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.053563:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510725248:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.053572:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1251:3654:x1871938510722432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6112us (7170us total) trans 4295070102 rc 0/0 00000100:00100000:3.0:1785220582.053579:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284557 00000100:00100000:0.0:1785220582.053586:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.053593:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510725248 00000100:00100000:3.0:1785220582.053596:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510725248 00000100:00100000:3.0:1785220582.053618:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284567 00000100:00100000:3.0:1785220582.053624:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+58:3655:x1871938510725248:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.053639:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29380 count 1 00010000:00010000:3.0:1785220582.053652:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.053664:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ea018105/0xf62499ab92791231 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f497 expref: 174 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.053672:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c4:0x0].0x0 since=20s v=(0 0) 00010000:00010000:3.0:1785220582.053676:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.053679:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.053698:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ea018105/0xf62499ab92791231 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f497 expref: 174 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.053698:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100491 00000100:00100000:2.0:1785220582.053705:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9acc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+174:2406:x1871938519969920:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.053708:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ea018105/0xf62499ab92791231 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f497 expref: 174 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053712:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.053717:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000094b9dd89/0xf62499ab92790e33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f443 expref: 174 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.053719:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0000_UUID:14194:1871938510725632:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.053726:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000094b9dd89/0xf62499ab92790e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f443 expref: 174 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.053737:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.053739:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094b9dd89/0xf62499ab92790e33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f443 expref: 174 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053748:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.053751:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.053769:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053776:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.053777:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100492 00000100:00100000:1.0:1785220582.053786:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb64000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+173:2407:x1871938519970048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.053794:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.053794:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.053799:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ce3fd24/0xf62499ab9279101d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f46d expref: 173 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.053808:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9acc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+173:2406:x1871938519969920:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (781us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.053808:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ce3fd24/0xf62499ab9279101d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f46d expref: 173 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.053815:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100491 00010000:00010000:1.0:1785220582.053821:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ce3fd24/0xf62499ab9279101d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f46d expref: 173 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053852:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053861:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053875:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053882:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.053895:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.053898:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.053901:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.053904:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.053916:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.053933:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.053948:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb64000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+171:2407:x1871938519970048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 163us (895us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.053954:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100492 00010000:00010000:1.0:1785220582.053972:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.053980:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.053983:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+59:3652:x1871938510722816:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7819us (7868us total) trans 4295070681 rc 0/0 00010000:00010000:1.0:1785220582.053988:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.053989:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284555 00010000:00010000:1.0:1785220582.053996:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054005:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.054007:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.054010:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054018:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.054022:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054030:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054043:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054050:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a53c1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7296:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054068:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.054071:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.054077:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+59:3652:x1871938510723072:12345-0@lo:6:osp-syn-0-0.0 Request processed in 6058us (7051us total) trans 4295070682 rc 0/0 00010000:00010000:2.0:1785220582.054078:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.054081:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054108:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054118:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054125:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a53b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054140:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.054143:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054149:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.054152:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054165:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054173:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054179:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a53cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054194:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.054196:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.054198:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.054208:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054216:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054224:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054231:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054244:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.054263:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284559 00010000:00010000:1.0:1785220582.054283:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.054288:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054295:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.054298:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054306:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.054315:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99919f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1251:3653:x1871938510722304:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6857us (8020us total) trans 4295070103 rc 0/0 00010000:00010000:1.0:1785220582.054318:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.054321:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284556 00010000:00010000:1.0:1785220582.054324:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a53ba lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7295:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054338:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.054340:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.054341:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.054346:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.054348:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054356:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.054356:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510725376:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.054364:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054371:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.054380:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.054383:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1785220582.054403:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f726680/2, rc 0, transno 4295070682 00000100:00100000:1.0:1785220582.054412:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f726680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510723072:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.054446:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbf480/2, rc 0, transno 4295070681 00000100:00100000:2.0:1785220582.054450:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1251:3652:x1871938510722176:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8843us (8916us total) trans 4295070104 rc 0/0 00000100:00100000:1.0:1785220582.054453:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510722816:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.054457:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284554 00000100:00100000:2.0:1785220582.054470:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510725376 00000100:00100000:2.0:1785220582.054473:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510725376 00000004:00080000:1.0:1785220582.054479:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99918a80/2, rc 0, transno 4295070104 00000100:00100000:1.0:1785220582.054486:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510722176:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.054498:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284568 00000100:00100000:2.0:1785220582.054504:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+59:3652:x1871938510725376:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.054506:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.054511:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510725504:0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.054516:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29381 count 1 00010000:00010000:2.0:1785220582.054528:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.054532:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.054545:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002d48a53c/0xf62499ab92791285 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4a5 expref: 170 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054552:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c5:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.054555:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.054559:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.054560:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99918000/2, rc 0, transno 4295070103 00000100:00100000:1.0:1785220582.054564:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510722304:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.054578:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002d48a53c/0xf62499ab92791285 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4a5 expref: 170 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.054578:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510725760:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.054588:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002d48a53c/0xf62499ab92791285 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4a5 expref: 170 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.054600:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510726016:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.054600:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.054621:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99919180/2, rc 0, transno 4295070102 00000100:00100000:1.0:1785220582.054625:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510722432:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.054633:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.054637:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510725888:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.054725:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.054734:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.054747:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.054765:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.054787:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.054898:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:2.0:1785220582.054901:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59300 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.054903:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054910:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.054913:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.054915:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510725504 00000100:00100000:2.0:1785220582.054918:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510725504 00010000:00010000:1.0:1785220582.054921:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054933:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.054937:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510725760 00000100:00100000:2.0:1785220582.054940:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510725760 00010000:00010000:1.0:1785220582.054946:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b07f87fc/0xf62499ab927a53ac lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7294:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.054954:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284569 00010000:00010000:1.0:1785220582.054959:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1785220582.054960:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1251:3652:x1871938510725504:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.054963:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054969:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00002000:00080000:2.0:1785220582.054972:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29339 count 1 00010000:00010000:1.0:1785220582.054972:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054979:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.054983:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054989:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.054996:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a53c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.055000:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ecbbcb2a/0xf62499ab9278f868 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f260 expref: 202 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.055007:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x729b:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.055009:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.055010:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.055011:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.055013:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.055014:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.055032:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ecbbcb2a/0xf62499ab9278f868 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f260 expref: 202 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.055042:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ecbbcb2a/0xf62499ab9278f868 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f260 expref: 202 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.055042:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.055053:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510726144:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.055071:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd590c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.055083:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510725888 00000100:00100000:2.0:1785220582.055141:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510725888 00000100:00100000:2.0:1785220582.055175:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284570 00000100:00100000:2.0:1785220582.055182:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+62:3654:x1871938510725760:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.055194:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29382 count 1 00010000:00010000:2.0:1785220582.055205:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.055210:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284571 00010000:00010000:2.0:1785220582.055217:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004c8f2f52/0xf62499ab92791357 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4b3 expref: 170 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.055218:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+62:3654:x1871938510725888:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.055225:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c6:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.055228:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c6:0x0].0x0 since=0s v=(1 0) 00002000:00080000:1.0:1785220582.055229:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29383 count 1 00010000:00010000:2.0:1785220582.055232:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.055239:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.055248:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004c8f2f52/0xf62499ab92791357 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4b3 expref: 170 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.055257:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004c8f2f52/0xf62499ab92791357 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4b3 expref: 170 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.055266:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510726272:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.055289:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59240 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.055292:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000641836ed/0xf62499ab92791430 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c1 expref: 170 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.055300:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c7:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.055303:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.055304:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.055306:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.055324:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000641836ed/0xf62499ab92791430 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c1 expref: 170 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.055333:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000641836ed/0xf62499ab92791430 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c1 expref: 170 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.055343:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510726400:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.055364:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.055527:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+62:3654:x1871938510721920:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7790us (9196us total) trans 4295070683 rc 0/0 00000100:00100000:0.0:1785220582.055535:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284558 00010000:00010000:0.0:1785220582.055564:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.055698:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510726528:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.055725:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.055788:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510726528 00000100:00100000:0.0:1785220582.055792:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510726528 00000100:00100000:0.0:1785220582.055820:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284572 00000100:00100000:0.0:1785220582.055826:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1252:3652:x1871938510726528:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.055839:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29340 count 1 00010000:00010000:0.0:1785220582.055852:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.055870:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000035fa0e4b/0xf62499ab9278f902 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f26e expref: 202 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.055877:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x729c:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220582.055880:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.055884:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.055904:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000035fa0e4b/0xf62499ab9278f902 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f26e expref: 202 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.055912:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000035fa0e4b/0xf62499ab9278f902 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f26e expref: 202 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.055922:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510726656:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.055946:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.055995:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.056002:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510726784:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.056025:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.056050:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510726784 00000100:00100000:2.0:1785220582.056053:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510726784 00000100:00100000:2.0:1785220582.056076:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284573 00000100:00100000:2.0:1785220582.056082:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1253:3652:x1871938510726784:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.056118:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29341 count 1 00010000:00010000:2.0:1785220582.056131:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.056143:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009d064e27/0xf62499ab9278fa05 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f27c expref: 202 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.056149:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x729d:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.056152:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.056155:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.056173:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009d064e27/0xf62499ab9278fa05 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f27c expref: 202 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.056181:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009d064e27/0xf62499ab9278fa05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f27c expref: 202 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.056190:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510726912:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.056216:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e82d19240 going to sleep for 11 seconds 00000004:00080000:1.0:1785220582.056366:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c0e00/2, rc 0, transno 4295070683 00000100:00100000:1.0:1785220582.056388:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510721920:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.059523:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:13820:1871938510724608:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.059550:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.060018:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0000_UUID:14194:1871938510725632:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.060037:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.060169:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.060353:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510724992:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.060370:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.060395:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510723840:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.060410:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.060618:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.060640:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.060669:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.060689:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.060710:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.060712:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970304 00000100:00100000:0.0:1785220582.060719:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970304 00010000:00010000:0.0:1785220582.060761:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c5255cb3/0xf62499ab9278f734 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f244 expref: 203 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.060788:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.060819:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.060843:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510724864:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.060860:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.061556:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.061581:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970560 00000100:00100000:0.0:1785220582.061586:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970560 00010000:00010000:0.0:1785220582.061609:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bb013b86/0xf62499ab9278f7dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f252 expref: 204 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.061717:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970816 00000100:00100000:0.0:1785220582.061720:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970816 00010000:00010000:0.0:1785220582.061736:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000111af165/0xf62499ab9278f638 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f228 expref: 205 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.061893:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970176 00000100:00100000:0.0:1785220582.061897:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970176 00010000:00010000:0.0:1785220582.061915:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000070389ebc/0xf62499ab92791197 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f489 expref: 171 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.061937:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970432 00000100:00100000:0.0:1785220582.061941:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970432 00010000:00010000:0.0:1785220582.061954:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002d48748d/0xf62499ab9278f6e0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f236 expref: 206 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.061973:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970688 00000100:00100000:0.0:1785220582.061977:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970688 00010000:00010000:0.0:1785220582.061990:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e24c0eee/0xf62499ab927910ef lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f47b expref: 172 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.062006:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971200 00000100:00100000:0.0:1785220582.062010:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971200 00010000:00010000:0.0:1785220582.062021:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ecbbcb2a/0xf62499ab9278f868 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f260 expref: 207 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.062036:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971072 00000100:00100000:0.0:1785220582.062038:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971072 00010000:00010000:0.0:1785220582.062049:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002d48a53c/0xf62499ab92791285 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4a5 expref: 173 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.062063:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100493 00000100:00100000:1.0:1785220582.062069:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510724736:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.062071:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+207:2406:x1871938519970304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.062096:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.062113:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c5255cb3/0xf62499ab9278f734 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f244 expref: 207 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062177:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.062215:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0000_UUID:14208:1871938510725120:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.062222:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c5255cb3/0xf62499ab9278f734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f244 expref: 207 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062228:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062246:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c5255cb3/0xf62499ab9278f734 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f244 expref: 207 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.062247:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100494 00000100:00100000:1.0:1785220582.062253:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+207:2406:x1871938519970560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.062261:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.062261:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7299:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.062265:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7299:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062266:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bb013b86/0xf62499ab9278f7dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f252 expref: 206 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062272:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062275:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bb013b86/0xf62499ab9278f7dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f252 expref: 206 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062281:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062287:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb013b86/0xf62499ab9278f7dc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f252 expref: 206 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062296:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062299:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062303:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062303:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.062311:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062329:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.062347:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+205:2406:x1871938519970560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (1730us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.062348:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+205:2406:x1871938519970304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 276us (2178us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.062355:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100494 00000100:00100000:0.0:1785220582.062363:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100493 00000100:00100000:1.0:1785220582.062371:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100495 00000100:00100000:0.0:1785220582.062375:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100496 00000100:00100000:1.0:1785220582.062377:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+203:2406:x1871938519970816:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.062380:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+173:2406:x1871938519970176:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.062384:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.062387:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.062389:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000111af165/0xf62499ab9278f638 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f228 expref: 203 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062391:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000070389ebc/0xf62499ab92791197 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f489 expref: 173 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062399:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000111af165/0xf62499ab9278f638 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f228 expref: 203 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062401:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000070389ebc/0xf62499ab92791197 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f489 expref: 173 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062412:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000111af165/0xf62499ab9278f638 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f228 expref: 203 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062414:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070389ebc/0xf62499ab92791197 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f489 expref: 173 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062423:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7297:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.062424:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062426:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7297:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.062427:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.062431:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062439:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062445:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062453:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062454:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.062471:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.062471:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+172:2406:x1871938519970176:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (1802us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.062478:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100496 00000100:00100000:1.0:1785220582.062485:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+202:2406:x1871938519970816:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (1845us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.062490:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100497 00000100:00100000:1.0:1785220582.062493:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100495 00000100:00100000:0.0:1785220582.062496:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+202:2406:x1871938519970432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.062504:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100498 00010000:00010000:0.0:1785220582.062504:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.062509:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90703050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+171:2406:x1871938519970688:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.062509:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002d48748d/0xf62499ab9278f6e0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f236 expref: 201 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062515:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.062520:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e24c0eee/0xf62499ab927910ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f47b expref: 171 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062520:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002d48748d/0xf62499ab9278f6e0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f236 expref: 201 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062530:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e24c0eee/0xf62499ab927910ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f47b expref: 171 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062533:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d48748d/0xf62499ab9278f6e0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f236 expref: 201 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062542:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e24c0eee/0xf62499ab927910ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f47b expref: 171 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062542:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7298:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.062546:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7298:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.062550:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062551:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062554:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062558:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062564:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062565:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.062580:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.062583:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.062596:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90703050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+170:2406:x1871938519970688:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 87us (1886us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.062596:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+200:2406:x1871938519970432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (1906us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.062603:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100497 00000100:00100000:1.0:1785220582.062708:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100498 00000100:00100000:2.0:1785220582.062728:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100499 00000100:00100000:2.0:1785220582.062735:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91582a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2406:x1871938519971200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.062738:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100500 00010000:00010000:2.0:1785220582.062744:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.062745:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91581f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+169:2406:x1871938519971072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.062749:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ecbbcb2a/0xf62499ab9278f868 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f260 expref: 199 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062751:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.062766:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002d48a53c/0xf62499ab92791285 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4a5 expref: 169 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.062770:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ecbbcb2a/0xf62499ab9278f868 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f260 expref: 199 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062776:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002d48a53c/0xf62499ab92791285 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4a5 expref: 169 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.062783:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.062786:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062786:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.062789:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.062791:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062793:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.062800:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062801:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.062813:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.062814:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.062830:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91582a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2406:x1871938519971200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (2041us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.062836:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100499 00010000:00010000:2.0:1785220582.062869:0:14160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ecbbcb2a/0xf62499ab9278f868 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f260 expref: 198 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.062886:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510726144:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.062896:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91581f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+169:2406:x1871938519971072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 152us (2076us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.062900:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.062906:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100500 00010000:00010000:2.0:1785220582.062909:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.062916:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.062925:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.062932:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.062949:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.063002:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063009:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063017:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063024:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063036:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.063246:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+1254:3652:x1871938510725504:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8283us (8709us total) trans 4295070105 rc 0/0 00000100:00100000:1.0:1785220582.063246:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+62:3654:x1871938510724096:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10715us (10761us total) trans 4295070684 rc 0/0 00000100:00100000:1.0:1785220582.063254:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284566 00010000:00010000:1.0:1785220582.063274:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.063280:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063288:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.063293:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.063293:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063303:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.063303:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.063311:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063317:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.063318:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063325:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5415 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.063333:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.063341:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.063344:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063351:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.063354:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063361:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063370:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063377:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5400 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063391:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.063394:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063401:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:0.0:1785220582.063401:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d86ecb850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1255:3654:x1871938510724480:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11419us (11608us total) trans 4295070106 rc 0/0 00010000:00010000:1.0:1785220582.063404:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063411:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.063424:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284569 00010000:00010000:1.0:1785220582.063427:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063433:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a53f9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063445:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.063447:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.063448:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.063457:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510726272:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.063462:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063470:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063470:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063479:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063491:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063497:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063504:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063505:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.063511:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063518:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063528:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.063574:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1255:3652:x1871938510724224:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11721us (11884us total) trans 4295070107 rc 0/0 00000100:00100000:1.0:1785220582.063581:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284563 00000100:00100000:2.0:1785220582.063588:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+62:3652:x1871938510724352:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11791us (11947us total) trans 4295070685 rc 0/0 00000100:00100000:2.0:1785220582.063594:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284562 00010000:00010000:1.0:1785220582.063596:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.063600:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063607:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.063607:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.063610:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063618:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063629:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063636:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a53eb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7299:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063649:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.063651:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.063653:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.063673:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510726400:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.063682:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063685:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063691:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.063694:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.063699:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.063702:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063713:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063720:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a53e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.063732:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510727168:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.063734:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.063736:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.063737:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.063767:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.063780:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.063790:0:14165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d48a53c/0xf62499ab92791285 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4a5 expref: 168 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.063801:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510726016:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.063808:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510727168 00010000:00010000:2.0:1785220582.063812:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.063812:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510727168 00010000:00010000:2.0:1785220582.063820:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063826:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063834:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063842:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.063854:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.063928:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+62:3652:x1871938510725376:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9424us (9547us total) trans 4295070686 rc 0/0 00000100:00100000:2.0:1785220582.063934:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284568 00000100:00100000:0.0:1785220582.063961:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284565 00000100:00100000:3.0:1785220582.064061:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.064273:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.064302:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.064351:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.065356:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284574 00000100:00100000:2.0:1785220582.065377:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1255:3655:x1871938510727168:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.065410:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29342 count 1 00000100:00100000:3.0:1785220582.065434:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.065434:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.065451:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006c75c008/0xf62499ab9278fa13 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f28a expref: 197 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.065459:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x729e:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.065464:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.065468:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.065499:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006c75c008/0xf62499ab9278fa13 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f28a expref: 197 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.065512:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006c75c008/0xf62499ab9278fa13 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f28a expref: 197 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.065527:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99918e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510727296:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.065591:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59300 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.065689:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971328 00000100:00100000:2.0:1785220582.065696:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971328 00010000:00010000:2.0:1785220582.065723:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004c8f2f52/0xf62499ab92791357 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4b3 expref: 168 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.065747:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971584 00000100:00100000:2.0:1785220582.065750:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971584 00010000:00010000:1.0:1785220582.065782:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.065789:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.065798:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.065802:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.065810:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.065885:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.065893:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a540e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.065920:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.065922:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.065924:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.065934:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519970944 00000100:00100000:1.0:1785220582.065938:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519970944 00010000:00010000:1.0:1785220582.065957:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ea018105/0xf62499ab92791231 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f497 expref: 169 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.065978:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971456 00000100:00100000:1.0:1785220582.065981:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971456 00010000:00010000:1.0:1785220582.065995:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000641836ed/0xf62499ab92791430 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c1 expref: 170 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.066013:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971712 00000100:00100000:1.0:1785220582.066015:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971712 00010000:00010000:2.0:1785220582.066016:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000035fa0e4b/0xf62499ab9278f902 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f26e expref: 198 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066026:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009d064e27/0xf62499ab9278fa05 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f27c expref: 199 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.066029:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971840 00000100:00100000:2.0:1785220582.066032:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971840 00000100:00100000:1.0:1785220582.066039:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100501 00010000:00010000:2.0:1785220582.066041:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ca27c79e/0xf62499ab92790ec6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f451 expref: 171 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.066045:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+171:2406:x1871938519971328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220582.066054:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100502 00010000:00010000:1.0:1785220582.066055:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220582.066059:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+171:2404:x1871938519970944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.066060:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004c8f2f52/0xf62499ab92791357 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4b3 expref: 171 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066067:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.066070:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004c8f2f52/0xf62499ab92791357 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4b3 expref: 171 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066071:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ea018105/0xf62499ab92791231 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f497 expref: 171 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066080:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ea018105/0xf62499ab92791231 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f497 expref: 171 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066119:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea018105/0xf62499ab92791231 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f497 expref: 171 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066130:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066133:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066139:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066148:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066171:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.066197:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+170:2404:x1871938519970944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 138us (1925us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.066205:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100502 00000100:00100000:2.0:1785220582.066222:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100503 00000100:00100000:2.0:1785220582.066229:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91580e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+169:2404:x1871938519971456:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.066236:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.066241:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000641836ed/0xf62499ab92791430 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c1 expref: 169 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066250:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000641836ed/0xf62499ab92791430 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c1 expref: 169 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066266:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000641836ed/0xf62499ab92791430 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c1 expref: 169 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066275:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066277:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066280:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066287:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066304:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.066322:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91580e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+168:2404:x1871938519971456:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (2020us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.066328:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100503 00000100:00100000:2.0:1785220582.066342:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100504 00000100:00100000:2.0:1785220582.066349:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91580700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2406:x1871938519971584:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.066357:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.066362:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000035fa0e4b/0xf62499ab9278f902 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f26e expref: 199 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066372:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000035fa0e4b/0xf62499ab9278f902 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f26e expref: 199 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066392:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066395:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066399:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066406:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066417:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.066433:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91580700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2406:x1871938519971584:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (2375us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.066439:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100504 00000100:00100000:2.0:1785220582.066454:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100505 00000100:00100000:2.0:1785220582.066460:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce71c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+198:2407:x1871938519971712:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.066467:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.066472:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009d064e27/0xf62499ab9278fa05 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f27c expref: 198 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066480:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009d064e27/0xf62499ab9278fa05 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f27c expref: 198 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066492:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066495:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.066499:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066506:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066517:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.066582:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c8f2f52/0xf62499ab92791357 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4b3 expref: 167 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066594:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.066597:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.066601:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.066609:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce71c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+198:2407:x1871938519971712:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 149us (2257us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.066609:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066628:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.066649:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+166:2406:x1871938519971328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 603us (5096us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.066656:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100501 00000100:00100000:1.0:1785220582.066669:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100506 00000100:00100000:1.0:1785220582.066675:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+165:2407:x1871938519971840:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.066683:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220582.066686:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100505 00010000:00010000:1.0:1785220582.066687:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ca27c79e/0xf62499ab92790ec6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f451 expref: 165 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066697:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ca27c79e/0xf62499ab92790ec6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f451 expref: 165 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066713:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.066716:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.066720:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066724:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066727:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066733:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066739:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.066743:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.066761:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.066765:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+165:2407:x1871938519971840:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (1320us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.066771:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100506 00010000:00010000:2.0:1785220582.066778:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.066794:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.066798:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066806:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.066809:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066817:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066839:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066850:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5407 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.066865:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.066867:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.066869:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.066890:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.066900:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510727040:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.066935:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.066945:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+62:3655:x1871938510725248:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13321us (13358us total) trans 4295070687 rc 0/0 00000100:00100000:2.0:1785220582.066952:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284567 00000100:00100000:2.0:1785220582.066968:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510727040 00000100:00100000:2.0:1785220582.066972:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510727040 00000004:00080000:1.0:1785220582.066972:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be7480/2, rc 0, transno 4295070687 00000100:00100000:1.0:1785220582.066984:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510725248:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.066995:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284575 00000100:00100000:2.0:1785220582.067000:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+62:3655:x1871938510727040:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.067015:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29384 count 1 00010000:00010000:2.0:1785220582.067031:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.067032:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220582.067040:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510727424:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.067043:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007f2c4cfd/0xf62499ab9279151e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4cf expref: 164 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067049:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c8:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.067052:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.067056:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.067064:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.067076:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007f2c4cfd/0xf62499ab9279151e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4cf expref: 164 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067104:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007f2c4cfd/0xf62499ab9279151e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4cf expref: 164 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.067116:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0000_UUID:14194:1871938510727552:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.067151:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd590c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.067166:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510727424 00000100:00100000:2.0:1785220582.067169:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510727424 00000100:00100000:2.0:1785220582.067193:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284576 00000100:00100000:2.0:1785220582.067199:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+63:3653:x1871938510727424:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.067212:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29385 count 1 00010000:00010000:2.0:1785220582.067224:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067236:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000051ac4919/0xf62499ab927915e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4dd expref: 164 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067243:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72c9:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.067246:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.067249:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067268:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000051ac4919/0xf62499ab927915e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4dd expref: 164 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.067270:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724a80/2, rc 0, transno 4295070684 00000100:00100000:1.0:1785220582.067277:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510724096:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.067278:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000051ac4919/0xf62499ab927915e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4dd expref: 164 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.067288:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510727680:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.067304:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510727808:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.067306:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a4d5600 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.067322:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.067325:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.067330:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067345:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067352:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.067354:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94e07480/2, rc 0, transno 4295070106 00000100:00100000:1.0:1785220582.067363:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94e07480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510724480:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.067364:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.067453:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+64:3654:x1871938510725760:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12271us (12851us total) trans 4295070688 rc 0/0 00000100:00100000:2.0:1785220582.067459:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284570 00000100:00100000:2.0:1785220582.067471:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510727808 00000100:00100000:2.0:1785220582.067474:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510727808 00000100:00100000:2.0:1785220582.067495:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284577 00000100:00100000:2.0:1785220582.067501:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+64:3653:x1871938510727808:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.067513:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29386 count 1 00010000:00010000:2.0:1785220582.067524:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.067531:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb66300/2, rc 0, transno 4295070688 00010000:00010000:2.0:1785220582.067536:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009e778e4b/0xf62499ab92791691 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4eb expref: 164 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.067539:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510725760:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.067543:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ca:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.067546:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.067550:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.067563:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.067570:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510727936:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.067574:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009e778e4b/0xf62499ab92791691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4eb expref: 164 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.067583:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009e778e4b/0xf62499ab92791691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4eb expref: 164 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.067591:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.067592:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510728064:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.067609:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba869480 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.067610:0:14250:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510727936 00000100:00100000:1.0:1785220582.067613:0:14250:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510727936 00000100:00100000:1.0:1785220582.067636:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284578 00000100:00100000:1.0:1785220582.067642:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+65:3655:x1871938510727936:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.067653:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29387 count 1 00010000:00010000:1.0:1785220582.067665:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067677:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e394c0ae/0xf62499ab9279171d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f9 expref: 164 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067683:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72cb:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.067686:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.067689:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067708:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e394c0ae/0xf62499ab9279171d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f9 expref: 164 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067717:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e394c0ae/0xf62499ab9279171d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f9 expref: 164 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.067727:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510728192:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.067745:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.067767:0:14222:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000035fa0e4b/0xf62499ab9278f902 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f26e expref: 197 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.067786:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510726656:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.067804:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067812:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067819:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067827:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067835:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067856:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.067887:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.067893:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067901:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.067904:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067912:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067924:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067931:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a541c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.067945:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1255:3652:x1871938510726528:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12118us (12219us total) trans 4295070108 rc 0/0 00010000:00010000:1.0:1785220582.067947:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.067950:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.067951:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284572 00010000:00010000:1.0:1785220582.067957:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.067960:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067967:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067974:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067982:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067982:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.067990:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a542a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067992:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.067999:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068002:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.068003:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.068005:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.068010:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.068014:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.068017:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068023:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.068026:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068033:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068042:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068049:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a53dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7297:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068065:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.068066:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.068068:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.068075:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1255:3655:x1871938510722560:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16831us (17236us total) trans 4295070109 rc 0/0 00000100:00100000:0.0:1785220582.068081:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284561 00000004:00080000:0.0:1785220582.068273:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7c700/2, rc 0, transno 4295070108 00000100:00100000:0.0:1785220582.068279:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510726528:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.068289:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e8601dc00/2, rc 0, transno 4295070105 00000100:00100000:0.0:1785220582.068292:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510725504:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.068300:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be6d80/2, rc 0, transno 4295070686 00000100:00100000:0.0:1785220582.068308:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510725376:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.068336:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991bb80/2, rc 0, transno 4295070109 00000100:00100000:1.0:1785220582.068342:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510722560:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.068345:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c3b80/2, rc 0, transno 4295070107 00000100:00100000:0.0:1785220582.068348:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510724224:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.068352:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510728320:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.068356:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724e00/2, rc 0, transno 4295070685 00000100:00100000:0.0:1785220582.068362:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510724352:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.068375:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.068390:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510728448:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.068412:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.068444:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510728320 00000100:00100000:1.0:1785220582.068448:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510728320 00000100:00100000:1.0:1785220582.068470:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510728448 00000100:00100000:1.0:1785220582.068472:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510728448 00000100:00100000:1.0:1785220582.068490:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284579 00000100:00100000:0.0:1785220582.068494:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510728960:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.068496:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+67:3655:x1871938510728320:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.068508:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29388 count 1 00000100:00100000:0.0:1785220582.068515:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.068520:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.068533:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510728960 00010000:00010000:1.0:1785220582.068534:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001889f5cf/0xf62499ab927917a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f507 expref: 164 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.068535:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510728960 00010000:00010000:1.0:1785220582.068541:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72cc:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.068543:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.068547:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.068555:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284580 00000100:00100000:0.0:1785220582.068560:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+67:3652:x1871938510728448:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.068570:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29389 count 1 00010000:00010000:1.0:1785220582.068571:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001889f5cf/0xf62499ab927917a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f507 expref: 164 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068580:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001889f5cf/0xf62499ab927917a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f507 expref: 164 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.068581:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.068591:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510729216:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.068592:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000322c7b8f/0xf62499ab92791851 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f515 expref: 164 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.068598:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72cd:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.068601:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.068604:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.068609:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.068626:0:13869:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284581 00000100:00100000:1.0:1785220582.068633:0:13869:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1255:3652:x1871938510728960:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.068645:0:13869:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29346 count 1 00010000:00010000:1.0:1785220582.068656:0:13869:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068668:0:13869:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009fa867f7/0xf62499ab9278fcd6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c2 expref: 196 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068675:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a2:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.068678:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.068682:0:13869:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.068695:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000322c7b8f/0xf62499ab92791851 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f515 expref: 164 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068697:0:13869:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009fa867f7/0xf62499ab9278fcd6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c2 expref: 196 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.068705:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000322c7b8f/0xf62499ab92791851 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f515 expref: 164 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.068712:0:13869:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009fa867f7/0xf62499ab9278fcd6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c2 expref: 196 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.068715:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510729344:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.068722:0:13869:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510729472:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.068732:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.068734:0:13869:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bad80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.068749:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510728576:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.068784:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.068808:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510729088:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.068825:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510728576 00000100:00100000:2.0:1785220582.068829:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510728576 00000100:00100000:1.0:1785220582.068839:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.068895:0:14200:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510729088 00000100:00100000:2.0:1785220582.068899:0:14200:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510729088 00000100:00100000:3.0:1785220582.068911:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284582 00000100:00100000:2.0:1785220582.068916:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284583 00000100:00100000:3.0:1785220582.068921:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1257:3653:x1871938510728576:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.068921:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1257:3653:x1871938510729088:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.068933:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29343 count 1 00002000:00080000:2.0:1785220582.068933:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29347 count 1 00010000:00010000:3.0:1785220582.068945:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.068945:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.068958:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c283de15/0xf62499ab9278fa83 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f298 expref: 196 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.068958:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000f0024a6/0xf62499ab9278fd62 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d0 expref: 196 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.068966:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x729f:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.068966:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a3:0x0].0x0 since=21s v=(0 0) 00010000:00010000:3.0:1785220582.068969:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.068969:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.068974:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.068974:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.068992:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c283de15/0xf62499ab9278fa83 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f298 expref: 196 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.068993:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000f0024a6/0xf62499ab9278fd62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d0 expref: 196 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.069003:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c283de15/0xf62499ab9278fa83 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f298 expref: 196 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069003:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000f0024a6/0xf62499ab9278fd62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d0 expref: 196 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.069013:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510729728:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.069018:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91582a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510729600:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.069038:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.069038:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e85436c00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.069066:0:14210:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009d064e27/0xf62499ab9278fa05 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f27c expref: 196 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.069078:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510726912:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.069110:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069120:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069128:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069137:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069148:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069162:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.069276:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1258:3652:x1871938510726784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13193us (13248us total) trans 4295070110 rc 0/0 00000100:00100000:2.0:1785220582.069283:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284573 00010000:00010000:2.0:1785220582.069320:0:14172:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca27c79e/0xf62499ab92790ec6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f451 expref: 164 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.069332:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0000_UUID:14172:1871938510723712:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.069345:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069353:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069359:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069366:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069372:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069387:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.069466:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+68:3655:x1871938510722048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18492us (18724us total) trans 4295070689 rc 0/0 00000100:00100000:2.0:1785220582.069473:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284560 00010000:00010000:2.0:1785220582.069491:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.069496:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069503:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.069507:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069516:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069529:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069536:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a5431 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069565:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.069569:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069580:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.069583:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069590:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069601:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069608:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a53d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069695:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.069697:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.069699:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.069718:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.069731:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069739:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069746:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069767:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069785:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.069859:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1258:3654:x1871938510723968:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17927us (18119us total) trans 4295070111 rc 0/0 00000100:00100000:2.0:1785220582.069866:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284564 00010000:00010000:2.0:1785220582.069960:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069969:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069976:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069982:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.069995:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.070075:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+68:3654:x1871938510725888:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14857us (15030us total) trans 4295070690 rc 0/0 00000100:00100000:2.0:1785220582.070081:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284571 00010000:00010000:1.0:1785220582.070279:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.070284:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070292:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.070295:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070302:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070314:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070321:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a53f2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7298:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070337:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.070340:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070346:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.070349:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070356:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070366:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070373:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5423 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.070385:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.070387:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.070389:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.070394:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.070436:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510728832:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.070468:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.070553:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbf100/2, rc 0, transno 4295070689 00000100:00100000:1.0:1785220582.070571:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510722048:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.070581:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.070587:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510728704:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.070611:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.070644:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb67800/2, rc 0, transno 4295070690 00000100:00100000:1.0:1785220582.070649:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510725888:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.070659:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99919c00/2, rc 0, transno 4295070111 00000100:00100000:1.0:1785220582.070667:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99919c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510723968:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.072182:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7d880/2, rc 0, transno 4295070110 00000100:00100000:1.0:1785220582.072192:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510726784:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.072212:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510729984:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.072254:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.072284:0:14203:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510728832 00000100:00100000:1.0:1785220582.072289:0:14203:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510728832 00000100:00100000:1.0:1785220582.072332:0:14203:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510728704 00000100:00100000:1.0:1785220582.072335:0:14203:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510728704 00000100:00100000:1.0:1785220582.072352:0:14203:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510729984 00000100:00100000:1.0:1785220582.072354:0:14203:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510729984 00000100:00100000:1.0:1785220582.072373:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284584 00000100:00100000:1.0:1785220582.072380:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1259:3655:x1871938510728832:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.072405:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29345 count 1 00010000:00010000:1.0:1785220582.072427:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072446:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e891a1cc/0xf62499ab9278fbf6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2b4 expref: 195 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072460:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a1:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.072464:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.072469:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072495:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e891a1cc/0xf62499ab9278fbf6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2b4 expref: 195 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072513:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e891a1cc/0xf62499ab9278fbf6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2b4 expref: 195 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.072525:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510730112:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.072581:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.072583:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284585 00000100:00100000:2.0:1785220582.072592:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1259:3655:x1871938510728704:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.072594:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284586 00000100:00100000:1.0:1785220582.072600:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+68:3655:x1871938510729984:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.072605:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29344 count 1 00002000:00080000:1.0:1785220582.072611:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29391 count 1 00010000:00010000:1.0:1785220582.072624:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072626:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072638:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000590d553c/0xf62499ab9278fb5c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2a6 expref: 195 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072638:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c1aef5ee/0xf62499ab92791946 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f531 expref: 163 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072646:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a0:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.072646:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72cf:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.072649:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.072649:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.072653:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072654:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072672:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000590d553c/0xf62499ab9278fb5c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2a6 expref: 195 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072673:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c1aef5ee/0xf62499ab92791946 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f531 expref: 163 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072683:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000590d553c/0xf62499ab9278fb5c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2a6 expref: 195 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.072684:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c1aef5ee/0xf62499ab92791946 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f531 expref: 163 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.072694:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510730240:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.072695:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0000_UUID:14221:1871938510730368:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.072714:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.072721:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e82d19240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.072731:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.072737:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510729856:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.072740:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.072792:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.072847:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510729856 00000100:00100000:2.0:1785220582.072851:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510729856 00000100:00100000:2.0:1785220582.072888:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284587 00000100:00100000:2.0:1785220582.072894:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+69:3655:x1871938510729856:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.072906:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29390 count 1 00010000:00010000:2.0:1785220582.072919:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072931:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000472aeb1a/0xf62499ab927918c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f523 expref: 163 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072938:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ce:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.072941:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.072945:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072965:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000472aeb1a/0xf62499ab927918c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f523 expref: 163 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.072974:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000472aeb1a/0xf62499ab927918c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f523 expref: 163 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.072985:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510730496:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.073010:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.073218:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510730752:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.073244:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.073264:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.073270:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94acbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510730624:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.073288:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.073370:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510727680:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.073389:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.073415:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0000_UUID:14194:1871938510727552:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.073426:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.073447:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510729728:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.073450:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91582a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510729600:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.073458:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.073463:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.073477:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99918e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510727296:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.073489:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.073511:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510728064:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.073523:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.073548:0:13869:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1eea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510729472:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.073560:0:13869:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.073579:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510730752 00000100:00100000:0.0:1785220582.073580:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510729344:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.073583:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510730752 00010000:00010000:0.0:1785220582.073592:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.073615:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510730624 00000100:00100000:1.0:1785220582.073618:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510730624 00000100:00100000:1.0:1785220582.073638:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284588 00000100:00100000:1.0:1785220582.073645:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1261:3653:x1871938510730752:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.073658:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29349 count 1 00010000:00010000:1.0:1785220582.073672:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.073682:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510728192:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.073684:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000075620d77/0xf62499ab9278fe26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ec expref: 195 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.073691:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a5:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.073694:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.073695:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.073698:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.073719:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000075620d77/0xf62499ab9278fe26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ec expref: 195 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.073729:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000075620d77/0xf62499ab9278fe26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ec expref: 195 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.073739:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284589 00000100:00100000:1.0:1785220582.073740:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0001_UUID:14163:1871938510730880:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.073746:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1261:3653:x1871938510730624:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.073769:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29348 count 1 00010000:00010000:2.0:1785220582.073781:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.073793:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000086f7b2cd/0xf62499ab9278fde0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2de expref: 195 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.073794:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bad80 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.073799:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a4:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.073803:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.073806:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.073815:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510729216:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.073824:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000086f7b2cd/0xf62499ab9278fde0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2de expref: 195 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.073832:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.073833:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000086f7b2cd/0xf62499ab9278fde0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2de expref: 195 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.073844:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510731008:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.073866:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.076949:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510730240:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.076979:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.078354:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078404:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078430:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078443:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078456:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078470:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078482:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078510:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078532:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078544:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078557:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078569:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078583:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078596:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.078616:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519971968 00000100:00100000:3.0:1785220582.078623:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519971968 00010000:00010000:3.0:1785220582.078665:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006c75c008/0xf62499ab9278fa13 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f28a expref: 196 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.078702:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972224 00000100:00100000:3.0:1785220582.078705:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972224 00000100:00100000:0.0:1785220582.078706:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972480 00000100:00100000:0.0:1785220582.078709:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972480 00010000:00010000:3.0:1785220582.078722:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000051ac4919/0xf62499ab927915e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4dd expref: 165 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078726:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e394c0ae/0xf62499ab9279171d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f9 expref: 165 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.078735:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972736 00000100:00100000:3.0:1785220582.078737:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972736 00000100:00100000:0.0:1785220582.078737:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972992 00000100:00100000:0.0:1785220582.078739:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972992 00010000:00010000:3.0:1785220582.078747:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000322c7b8f/0xf62499ab92791851 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f515 expref: 166 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078748:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000f0024a6/0xf62499ab9278fd62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d0 expref: 197 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.078771:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519973504 00000100:00100000:0.0:1785220582.078771:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519973248 00000100:00100000:3.0:1785220582.078773:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519973504 00000100:00100000:0.0:1785220582.078773:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519973248 00010000:00010000:3.0:1785220582.078786:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000472aeb1a/0xf62499ab927918c8 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f523 expref: 168 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078786:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c1aef5ee/0xf62499ab92791946 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f531 expref: 168 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.078797:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519973632 00000100:00100000:3.0:1785220582.078799:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519973632 00000100:00100000:0.0:1785220582.078802:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519973760 00000100:00100000:0.0:1785220582.078804:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519973760 00010000:00010000:3.0:1785220582.078809:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e891a1cc/0xf62499ab9278fbf6 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2b4 expref: 199 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078814:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000075620d77/0xf62499ab9278fe26 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ec expref: 199 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.078819:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972096 00000100:00100000:3.0:1785220582.078821:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972096 00000100:00100000:0.0:1785220582.078824:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972352 00000100:00100000:0.0:1785220582.078826:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972352 00010000:00010000:0.0:1785220582.078836:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009e778e4b/0xf62499ab92791691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4eb expref: 170 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.078841:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007f2c4cfd/0xf62499ab9279151e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4cf expref: 170 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.078848:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972608 00000100:00100000:0.0:1785220582.078850:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972608 00000100:00100000:3.0:1785220582.078851:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519972864 00000100:00100000:3.0:1785220582.078853:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519972864 00010000:00010000:0.0:1785220582.078859:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001889f5cf/0xf62499ab927917a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f507 expref: 171 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.078862:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009fa867f7/0xf62499ab9278fcd6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c2 expref: 200 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.078870:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519973120 00000100:00100000:0.0:1785220582.078872:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519973120 00000100:00100000:3.0:1785220582.078876:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100507 00010000:00010000:0.0:1785220582.078880:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c283de15/0xf62499ab9278fa83 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f298 expref: 201 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.078884:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91582a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+201:2407:x1871938519971968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.078892:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100508 00000100:00100000:0.0:1785220582.078897:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91581f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+171:2407:x1871938519972224:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.078899:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.078905:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006c75c008/0xf62499ab9278fa13 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f28a expref: 201 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078905:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.078909:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000051ac4919/0xf62499ab927915e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4dd expref: 171 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.078916:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006c75c008/0xf62499ab9278fa13 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f28a expref: 201 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078923:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000051ac4919/0xf62499ab927915e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4dd expref: 171 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.078940:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006c75c008/0xf62499ab9278fa13 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f28a expref: 201 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078940:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051ac4919/0xf62499ab927915e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4dd expref: 171 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.078956:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.078957:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.078961:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.078961:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.078968:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078968:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.078978:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.078979:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079008:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.079011:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.079063:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91581f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+170:2407:x1871938519972224:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 164us (657us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.079072:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100508 00000100:00100000:1.0:1785220582.079214:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510730112:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.079236:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.079271:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0001_UUID:14163:1871938510730880:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.079282:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.079283:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100509 00000100:00100000:0.0:1785220582.079290:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+169:2407:x1871938519972480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.079298:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.079300:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100510 00010000:00010000:0.0:1785220582.079303:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e394c0ae/0xf62499ab9279171d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f9 expref: 169 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.079306:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91580700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+200:2407:x1871938519972992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.079313:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.079317:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000f0024a6/0xf62499ab9278fd62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d0 expref: 200 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079319:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e394c0ae/0xf62499ab9279171d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f9 expref: 169 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.079327:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000f0024a6/0xf62499ab9278fd62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d0 expref: 200 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079332:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e394c0ae/0xf62499ab9279171d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f9 expref: 169 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.079339:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f0024a6/0xf62499ab9278fd62 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2d0 expref: 200 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079341:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.079344:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.079348:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.079348:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.079351:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.079355:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079356:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.079364:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079372:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.079383:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.079427:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0000_UUID:14221:1871938510730368:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.079441:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91582a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+199:2407:x1871938519971968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 557us (1092us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.079441:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.079448:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100507 00000100:00100000:3.0:1785220582.079462:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100511 00000100:00100000:2.0:1785220582.079467:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510730496:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.079468:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91580e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+168:2407:x1871938519972736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.079476:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.079478:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079481:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000322c7b8f/0xf62499ab92791851 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f515 expref: 168 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079490:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000322c7b8f/0xf62499ab92791851 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f515 expref: 168 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.079494:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079503:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000322c7b8f/0xf62499ab92791851 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f515 expref: 168 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.079503:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079514:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.079514:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079517:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.079521:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.079522:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079529:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.079538:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.079543:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.079561:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91580e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+167:2407:x1871938519972736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (1116us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.079567:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100511 00000100:00100000:3.0:1785220582.079580:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100512 00000100:00100000:3.0:1785220582.079585:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+166:2407:x1871938519973504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.079593:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.079596:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000472aeb1a/0xf62499ab927918c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f523 expref: 166 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079605:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000472aeb1a/0xf62499ab927918c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f523 expref: 166 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079616:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000472aeb1a/0xf62499ab927918c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f523 expref: 166 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079625:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.079628:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.079632:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079638:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079655:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.079673:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+165:2407:x1871938519973504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 87us (1189us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.079679:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100512 00000100:00100000:3.0:1785220582.079693:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100513 00000100:00100000:0.0:1785220582.079696:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+164:2407:x1871938519972480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 407us (1265us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.079700:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+164:2407:x1871938519973248:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.079702:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100509 00010000:00010000:3.0:1785220582.079708:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.079712:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c1aef5ee/0xf62499ab92791946 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f531 expref: 163 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.079713:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100514 00000100:00100000:0.0:1785220582.079719:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+198:2406:x1871938519973632:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.079722:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c1aef5ee/0xf62499ab92791946 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f531 expref: 163 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079725:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.079729:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e891a1cc/0xf62499ab9278fbf6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2b4 expref: 198 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079734:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1aef5ee/0xf62499ab92791946 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f531 expref: 163 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079743:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.079745:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e891a1cc/0xf62499ab9278fbf6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2b4 expref: 198 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079746:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.079750:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079770:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079770:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e891a1cc/0xf62499ab9278fbf6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2b4 expref: 198 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079781:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.079784:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.079788:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.079788:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.079795:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.079804:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+162:2407:x1871938519973248:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (1333us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.079810:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100513 00010000:00010000:0.0:1785220582.079810:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.079827:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2406:x1871938519973632:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (1316us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.079900:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91580700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2407:x1871938519972992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 593us (1442us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.079920:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100515 00000100:00100000:3.0:1785220582.079928:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+197:2406:x1871938519973760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.079936:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.079940:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000075620d77/0xf62499ab9278fe26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ec expref: 197 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.079944:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+70:3653:x1871938510727424:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12745us (12878us total) trans 4295070691 rc 0/0 00010000:00010000:3.0:1785220582.079950:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000075620d77/0xf62499ab9278fe26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ec expref: 197 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.079951:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284576 00010000:00010000:3.0:1785220582.079962:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075620d77/0xf62499ab9278fe26 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2ec expref: 197 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079972:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.079972:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079975:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.079980:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.079980:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.079988:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.079990:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.079998:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.080004:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.080015:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.080018:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+196:2406:x1871938519973760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (1485us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.080134:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+70:3655:x1871938510729856:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7228us (7329us total) trans 4295070692 rc 0/0 00000100:00100000:2.0:1785220582.080141:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284587 00000100:00100000:0.0:1785220582.080263:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100514 00000100:00100000:0.0:1785220582.080282:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100516 00000100:00100000:0.0:1785220582.080289:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+161:2407:x1871938519972352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.080295:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100510 00010000:00010000:0.0:1785220582.080297:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.080302:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009e778e4b/0xf62499ab92791691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4eb expref: 161 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080317:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009e778e4b/0xf62499ab92791691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4eb expref: 161 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080328:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100517 00010000:00010000:0.0:1785220582.080329:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e778e4b/0xf62499ab92791691 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4eb expref: 161 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080335:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+161:2407:x1871938519972096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.080339:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.080342:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.080342:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.080346:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080347:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007f2c4cfd/0xf62499ab9279151e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4cf expref: 160 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080354:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080357:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007f2c4cfd/0xf62499ab9279151e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4cf expref: 160 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080370:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f2c4cfd/0xf62499ab9279151e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4cf expref: 160 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080371:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.080380:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.080384:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72c8:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.080385:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+159:2407:x1871938519972352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (1827us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.080388:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.080391:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100516 00010000:00010000:1.0:1785220582.080396:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.080403:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100518 00000100:00100000:0.0:1785220582.080409:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2407:x1871938519972608:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.080416:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.080419:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.080421:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001889f5cf/0xf62499ab927917a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f507 expref: 158 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080431:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2407:x1871938519972096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (1886us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.080435:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001889f5cf/0xf62499ab927917a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f507 expref: 158 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080437:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100517 00010000:00010000:0.0:1785220582.080447:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001889f5cf/0xf62499ab927917a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f507 expref: 157 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080449:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100519 00000100:00100000:1.0:1785220582.080455:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e11050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+194:2407:x1871938519972864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.080456:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.080459:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.080462:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.080463:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080467:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009fa867f7/0xf62499ab9278fcd6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c2 expref: 194 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080471:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080476:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009fa867f7/0xf62499ab9278fcd6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c2 expref: 194 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080486:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.080488:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fa867f7/0xf62499ab9278fcd6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2c2 expref: 194 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080498:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a2:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.080498:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+156:2407:x1871938519972608:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (1928us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.080501:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a2:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.080505:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100518 00010000:00010000:1.0:1785220582.080506:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080513:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.080516:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100520 00000100:00100000:0.0:1785220582.080522:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ac700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+193:2407:x1871938519973120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.080529:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.080534:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.080534:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c283de15/0xf62499ab9278fa83 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f298 expref: 193 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080545:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e11050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+193:2407:x1871938519972864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (1962us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.080548:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c283de15/0xf62499ab9278fa83 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f298 expref: 193 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080552:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100519 00010000:00010000:0.0:1785220582.080561:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c283de15/0xf62499ab9278fa83 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f298 expref: 192 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080570:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080570:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.080573:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x729f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.080578:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080579:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080586:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080587:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080595:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080607:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1785220582.080610:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.080698:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ac700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+191:2407:x1871938519973120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 176us (2100us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.080705:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100520 00010000:00010000:0.0:1785220582.080726:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.080728:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100515 00010000:00010000:0.0:1785220582.080735:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080742:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080750:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.080760:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.080770:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.080776:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:3.0:1785220582.080783:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.080790:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.080797:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00010000:00010000:2.0:1785220582.080868:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1785220582.080874:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080884:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.080888:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080897:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080911:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080918:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5446 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080934:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220582.080937:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080948:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.080951:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080958:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.080960:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510729088:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12038us (12119us total) trans 4295070112 rc 0/0 00000100:00100000:1.0:1785220582.080967:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284583 00010000:00010000:2.0:1785220582.080969:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080975:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5493 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.080984:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.080987:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.080988:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080990:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.080997:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.080997:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.081000:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081001:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081009:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081010:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081018:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081019:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081027:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5462 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081027:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5470 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081040:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.081041:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.081043:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.081044:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081044:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081051:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.081054:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081060:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081068:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081074:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5477 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.081075:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510731008:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.081109:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081128:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081130:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081132:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081142:0:13869:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081150:0:13869:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081158:0:13869:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081166:0:13869:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081180:0:13869:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.081235:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+70:3652:x1871938510728448:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12674us (12821us total) trans 4295070693 rc 0/0 00000100:00100000:0.0:1785220582.081242:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284580 00000100:00100000:1.0:1785220582.081249:0:13869:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1263:3652:x1871938510728960:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12618us (12734us total) trans 4295070114 rc 0/0 00000100:00100000:1.0:1785220582.081255:0:13869:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284581 00010000:00010000:1.0:1785220582.081271:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.081272:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081281:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.081281:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.081290:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081294:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.081297:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081301:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081309:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1785220582.081309:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.081377:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+71:3655:x1871938510728320:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12880us (12999us total) trans 4295070695 rc 0/0 00000100:00100000:0.0:1785220582.081377:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+71:3655:x1871938510727936:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13734us (13783us total) trans 4295070694 rc 0/0 00000100:00100000:0.0:1785220582.081383:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284578 00010000:00010000:2.0:1785220582.081415:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.081421:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081428:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.081430:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081437:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081449:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081455:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5469 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081476:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.081480:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081487:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.081489:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081497:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081508:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.081509:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284579 00010000:00010000:2.0:1785220582.081515:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5454 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081534:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.081537:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081544:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.081547:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081559:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.081564:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1262:3653:x1871938510728576:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12646us (12779us total) trans 4295070113 rc 0/0 00010000:00010000:2.0:1785220582.081568:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.081572:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284582 00010000:00010000:2.0:1785220582.081575:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a545b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081603:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081612:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081620:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081628:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081634:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081646:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.081732:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.081737:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081744:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.081759:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.081766:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+1262:3653:x1871938510730752:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8121us (8521us total) trans 4295070115 rc 0/0 00010000:00010000:2.0:1785220582.081767:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.081774:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284588 00010000:00010000:2.0:1785220582.081779:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081786:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a549a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081789:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081797:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081804:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.081805:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081806:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081807:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.081814:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081817:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081821:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081825:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081833:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081840:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081852:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.081909:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.081929:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+70:3655:x1871938510729984:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9328us (9674us total) trans 4295070696 rc 0/0 00000100:00100000:2.0:1785220582.081936:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284586 00010000:00010000:2.0:1785220582.081952:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.081956:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081963:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.081967:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.081974:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+70:3655:x1871938510727040:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14973us (15038us total) trans 4295070697 rc 0/0 00010000:00010000:2.0:1785220582.081976:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.081980:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284575 00010000:00010000:2.0:1785220582.081986:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.081993:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5485 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.081996:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.081999:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082006:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.082006:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.082008:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.082009:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.082009:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082019:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082022:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082028:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082033:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082035:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082040:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a543f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082043:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082053:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.082054:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1785220582.082056:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082061:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.082064:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082071:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082081:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.082224:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94acb800/2, rc 0, transno 4295070115 00000100:00100000:0.0:1785220582.082237:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510730752:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.082264:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:0.0:1785220582.082293:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7e680/2, rc 0, transno 4295070113 00000100:00100000:0.0:1785220582.082299:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510728576:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.082324:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99be4e00/2, rc 0, transno 4295070112 00000100:00100000:0.0:1785220582.082331:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99be4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510729088:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.082358:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb64000/2, rc 0, transno 4295070691 00000100:00100000:0.0:1785220582.082367:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510727424:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.082373:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.082385:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510731392:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.082414:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.082532:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5438 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x729e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082559:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.082561:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.082563:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.082569:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.082579:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082588:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082596:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.082603:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.082607:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1262:3655:x1871938510727168:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17238us (18826us total) trans 4295070116 rc 0/0 00000100:00100000:2.0:1785220582.082716:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284574 00000100:00100000:2.0:1785220582.082732:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510731392 00000100:00100000:2.0:1785220582.082737:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510731392 00000100:00100000:2.0:1785220582.082780:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284590 00000100:00100000:2.0:1785220582.082787:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1262:3653:x1871938510731392:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.082811:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29352 count 1 00010000:00010000:1.0:1785220582.082817:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.082829:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082844:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006061f63d/0xf62499ab92790072 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f316 expref: 189 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082852:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a8:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.082856:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.082861:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082885:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006061f63d/0xf62499ab92790072 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f316 expref: 189 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.082896:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006061f63d/0xf62499ab92790072 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f316 expref: 189 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.082908:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510731520:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.082944:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.082944:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d63c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.082979:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.082983:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.082990:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.082993:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083002:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083015:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083022:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a547e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083041:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.083042:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.083044:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.083047:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+1263:3655:x1871938510728832:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10667us (12578us total) trans 4295070117 rc 0/0 00000100:00100000:1.0:1785220582.083054:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284584 00000100:00100000:0.0:1785220582.083056:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519973376 00000100:00100000:0.0:1785220582.083059:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519973376 00000100:00100000:1.0:1785220582.083071:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519973888 00000100:00100000:1.0:1785220582.083074:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519973888 00010000:00010000:0.0:1785220582.083078:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000590d553c/0xf62499ab9278fb5c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2a6 expref: 191 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.083140:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100521 00000100:00100000:0.0:1785220582.083147:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+191:2407:x1871938519973376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.083159:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.083164:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000590d553c/0xf62499ab9278fb5c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2a6 expref: 191 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083175:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000590d553c/0xf62499ab9278fb5c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2a6 expref: 191 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083190:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000590d553c/0xf62499ab9278fb5c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2a6 expref: 191 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083199:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.083203:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.083207:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083221:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.083239:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.083253:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+190:2407:x1871938519973376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 107us (990us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.083260:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100521 00010000:00010000:1.0:1785220582.083471:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000086f7b2cd/0xf62499ab9278fde0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2de expref: 189 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.083496:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100522 00000100:00100000:1.0:1785220582.083505:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+189:2407:x1871938519973888:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.083513:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.083518:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000086f7b2cd/0xf62499ab9278fde0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2de expref: 189 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083527:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000086f7b2cd/0xf62499ab9278fde0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2de expref: 189 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083540:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000086f7b2cd/0xf62499ab9278fde0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2de expref: 189 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083550:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.083553:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.083557:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083565:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083584:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.083604:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+188:2407:x1871938519973888:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (661us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.083611:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100522 00010000:00010000:1.0:1785220582.083629:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083642:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083650:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.083658:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.083684:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.085130:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00000100:00100000:1.0:1785220582.085305:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1263:3653:x1871938510730624:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11557us (12013us total) trans 4295070118 rc 0/0 00000100:00100000:1.0:1785220582.085315:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284589 00010000:00010000:1.0:1785220582.085408:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.085422:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.085432:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.085436:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.085441:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7ea00/2, rc 0, transno 4295070114 00010000:00010000:2.0:1785220582.085445:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.085452:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510728960:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.085461:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.085464:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7e300/2, rc 0, transno 4295070693 00010000:00010000:2.0:1785220582.085468:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a54a1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.085471:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510728448:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.085481:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1785220582.085489:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.085490:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.085492:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.085505:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.085505:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510731648:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.085513:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.085527:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.085535:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.085536:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.085548:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.085561:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510731648 00000100:00100000:1.0:1785220582.085566:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510731648 00000100:00100000:1.0:1785220582.085598:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284591 00000100:00100000:1.0:1785220582.085606:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb21880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+70:3652:x1871938510731648:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.085628:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29392 count 1 00010000:00010000:1.0:1785220582.085647:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.085664:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d6ef86ed/0xf62499ab927919f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f53f expref: 155 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.085672:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d0:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.085676:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.085680:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.085703:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.085704:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d6ef86ed/0xf62499ab927919f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f53f expref: 155 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.085710:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.085716:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d6ef86ed/0xf62499ab927919f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f53f expref: 155 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.085717:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.085720:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.085728:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510731904:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.085728:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.085739:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.085745:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a548c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.085771:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.085774:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.085776:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.085782:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.085802:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94acbb80/2, rc 0, transno 4295070118 00000100:00100000:0.0:1785220582.085807:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94acbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510730624:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.085854:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1263:3655:x1871938510728704:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13263us (15241us total) trans 4295070119 rc 0/0 00000100:00100000:2.0:1785220582.085861:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284585 00010000:00010000:2.0:1785220582.085890:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.085899:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.085907:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.085914:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.085967:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.086050:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+71:3653:x1871938510727808:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18549us (18724us total) trans 4295070698 rc 0/0 00000100:00100000:2.0:1785220582.086061:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284577 00000100:00100000:3.0:1785220582.086080:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.086116:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510731520:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.086141:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.086161:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974016 00000100:00100000:2.0:1785220582.086164:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974016 00010000:00010000:2.0:1785220582.086181:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006061f63d/0xf62499ab92790072 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f316 expref: 188 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.086212:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100523 00000100:00100000:2.0:1785220582.086220:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+188:2407:x1871938519974016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.086230:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.086236:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006061f63d/0xf62499ab92790072 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f316 expref: 188 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.086245:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006061f63d/0xf62499ab92790072 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f316 expref: 188 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.086262:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006061f63d/0xf62499ab92790072 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f316 expref: 188 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.086272:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.086275:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.086280:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.086289:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.086313:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.086323:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.086330:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+187:2407:x1871938519974016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.086337:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100523 00000100:00100000:1.0:1785220582.086378:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510732160:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.086403:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.086430:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510731264:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.086451:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.086610:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.086681:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.086690:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.086693:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.086700:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.086711:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.086718:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a544d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.086732:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.086733:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.086735:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.086766:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbea00/2, rc 0, transno 4295070698 00000100:00100000:0.0:1785220582.086782:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510727808:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.086952:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.086961:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510732288:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.086989:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.087215:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510731136:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.087243:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.087931:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510732160 00000100:00100000:2.0:1785220582.087938:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510732160 00000100:00100000:1.0:1785220582.087943:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510732032:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.087973:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510731264 00000100:00100000:2.0:1785220582.087977:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510731264 00000100:00100000:2.0:1785220582.087994:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510732288 00000100:00100000:2.0:1785220582.087997:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510732288 00000100:00100000:2.0:1785220582.088029:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284592 00000100:00100000:2.0:1785220582.088037:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1264:3652:x1871938510732160:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.088061:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29354 count 1 00010000:00010000:2.0:1785220582.088083:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.088216:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284593 00000100:00100000:1.0:1785220582.088226:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1264:3655:x1871938510731264:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.088239:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29351 count 1 00010000:00010000:1.0:1785220582.088252:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088271:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000042f1236f/0xf62499ab9278ffbc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f308 expref: 186 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088280:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a7:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.088285:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.088289:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088316:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000042f1236f/0xf62499ab9278ffbc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f308 expref: 186 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088330:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000042f1236f/0xf62499ab9278ffbc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f308 expref: 186 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.088342:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510732416:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.088395:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.088553:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb22680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510731776:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.088580:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.088606:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510731136 00000100:00100000:1.0:1785220582.088611:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510731136 00000100:00100000:1.0:1785220582.088635:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284594 00000100:00100000:1.0:1785220582.088643:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a52b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+71:3652:x1871938510732288:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.088658:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29394 count 1 00010000:00010000:1.0:1785220582.088671:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088684:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c44910bd/0xf62499ab92791b3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f55b expref: 155 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088690:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d2:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.088695:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.088699:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.088715:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000d1243ac/0xf62499ab92790144 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f332 expref: 186 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088717:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c44910bd/0xf62499ab92791b3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f55b expref: 155 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.088724:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72aa:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.088727:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.088727:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c44910bd/0xf62499ab92791b3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f55b expref: 155 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.088731:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.088738:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510732544:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.088747:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000d1243ac/0xf62499ab92790144 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f332 expref: 186 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.088767:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000d1243ac/0xf62499ab92790144 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f332 expref: 186 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.088777:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.088778:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510732672:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.088799:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d63c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.088807:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510731776 00000100:00100000:1.0:1785220582.088810:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510731776 00000100:00100000:2.0:1785220582.088814:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284595 00000100:00100000:2.0:1785220582.088821:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1265:3654:x1871938510731136:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.088832:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29350 count 1 00010000:00010000:2.0:1785220582.088845:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.088857:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000816663e0/0xf62499ab9278fec0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2fa expref: 186 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.088864:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a6:0x0].0x0 since=21s v=(0 0) 00010000:00010000:2.0:1785220582.088866:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.088870:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.088889:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000816663e0/0xf62499ab9278fec0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2fa expref: 186 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.088902:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284596 00010000:00010000:2.0:1785220582.088904:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000816663e0/0xf62499ab9278fec0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2fa expref: 186 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.088909:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+72:3654:x1871938510731776:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.088913:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510732800:192.168.201.51@tcp:104: 00002000:00080000:1.0:1785220582.088920:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29393 count 1 00010000:00010000:1.0:1785220582.088931:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.088935:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.088943:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b50ed4e4/0xf62499ab92791a9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f54d expref: 155 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088951:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d1:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.088953:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.088957:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088974:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b50ed4e4/0xf62499ab92791a9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f54d expref: 155 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.088983:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b50ed4e4/0xf62499ab92791a9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f54d expref: 155 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.088993:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91583480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:13867:1871938510732928:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.089016:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.089033:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.089055:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510732032 00000100:00100000:1.0:1785220582.089059:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510732032 00000004:00080000:0.0:1785220582.089284:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083d880/2, rc 0, transno 4295070692 00000100:00100000:0.0:1785220582.089297:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510729856:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.089308:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083c380/2, rc 0, transno 4295070696 00000100:00100000:0.0:1785220582.089311:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510729984:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.089322:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7c000/2, rc 0, transno 4295070119 00000100:00100000:0.0:1785220582.089331:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510728704:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.089342:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7ed80/2, rc 0, transno 4295070117 00000100:00100000:0.0:1785220582.089346:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510728832:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.089353:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284597 00000004:00080000:0.0:1785220582.089355:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7f480/2, rc 0, transno 4295070695 00000100:00100000:0.0:1785220582.089358:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510728320:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.089363:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1266:3655:x1871938510732032:12345-0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.089367:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbc000/2, rc 0, transno 4295070694 00000100:00100000:0.0:1785220582.089370:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510727936:0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.089378:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29353 count 1 00000004:00080000:0.0:1785220582.089379:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9991a300/2, rc 0, transno 4295070697 00000100:00100000:0.0:1785220582.089382:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9991a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510727040:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.089389:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.089391:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7d180/2, rc 0, transno 4295070116 00000100:00100000:0.0:1785220582.089394:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510727168:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.089402:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a483a991/0xf62499ab9279010c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f324 expref: 186 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.089409:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72a9:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.089413:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.089416:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.089437:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a483a991/0xf62499ab9279010c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f324 expref: 186 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.089446:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a483a991/0xf62499ab9279010c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f324 expref: 186 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.089456:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510733056:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.089479:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.089570:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510733440:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.089600:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.089620:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.089626:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91580700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510733568:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.089648:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.089706:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510733952:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.089727:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.089742:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.089747:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510734080:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.090068:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.090119:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91580e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733312:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.090141:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.090162:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733824:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.090183:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.090206:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1785220582.090212:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733184:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.090232:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.090254:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510733440 00000100:00100000:0.0:1785220582.090254:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510733568 00000100:00100000:1.0:1785220582.090259:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510733440 00000100:00100000:0.0:1785220582.090259:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510733568 00000100:00100000:1.0:1785220582.090289:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510733952 00000100:00100000:1.0:1785220582.090292:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510733952 00000100:00100000:0.0:1785220582.090351:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510734080 00000100:00100000:0.0:1785220582.090355:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510734080 00000100:00100000:0.0:1785220582.090375:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510733312 00000100:00100000:0.0:1785220582.090377:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510733312 00000100:00100000:0.0:1785220582.090391:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510733824 00000100:00100000:0.0:1785220582.090394:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510733824 00000100:00100000:0.0:1785220582.090408:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510733184 00000100:00100000:0.0:1785220582.090411:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510733184 00000100:00100000:0.0:1785220582.090427:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284598 00000100:00100000:0.0:1785220582.090433:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+76:3652:x1871938510733440:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.090452:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29397 count 1 00010000:00010000:0.0:1785220582.090470:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090486:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000248a3d80/0xf62499ab92791cc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f585 expref: 155 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090495:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d5:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.090498:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.090502:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090526:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000248a3d80/0xf62499ab92791cc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f585 expref: 155 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.090533:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284599 00010000:00010000:0.0:1785220582.090537:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000248a3d80/0xf62499ab92791cc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f585 expref: 155 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.090543:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+76:3652:x1871938510733568:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.090548:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510734208:192.168.201.51@tcp:104: 00002000:00080000:3.0:1785220582.090554:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29398 count 1 00010000:00010000:3.0:1785220582.090565:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.090576:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc2f7c5c/0xf62499ab92791db4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f593 expref: 155 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.090583:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d6:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.090586:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.090590:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.090590:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.090605:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284600 00010000:00010000:3.0:1785220582.090608:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dc2f7c5c/0xf62499ab92791db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f593 expref: 155 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.090617:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dc2f7c5c/0xf62499ab92791db4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f593 expref: 155 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.090627:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510734336:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.090652:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.090671:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.090681:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.090691:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.090699:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.090717:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1269:3652:x1871938510733952:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.090732:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29356 count 1 00010000:00010000:0.0:1785220582.090746:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090768:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003a4b923c/0xf62499ab927902c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f34e expref: 186 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090774:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ac:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220582.090777:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.090781:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090801:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003a4b923c/0xf62499ab927902c5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f34e expref: 186 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090812:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003a4b923c/0xf62499ab927902c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f34e expref: 186 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.090822:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510734464:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.090846:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.090863:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284601 00000100:00100000:0.0:1785220582.090870:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1269:3652:x1871938510734080:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.090884:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29357 count 1 00010000:00010000:0.0:1785220582.090895:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090906:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f88cc130/0xf62499ab92790358 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f35c expref: 186 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090912:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ad:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220582.090915:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.090918:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090935:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f88cc130/0xf62499ab92790358 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f35c expref: 186 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.090944:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f88cc130/0xf62499ab92790358 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f35c expref: 186 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.090954:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510734592:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.090975:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464780 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.090992:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284602 00000100:00100000:0.0:1785220582.090999:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+76:3655:x1871938510733312:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.091009:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29396 count 1 00010000:00010000:0.0:1785220582.091020:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091038:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d307b166/0xf62499ab92791c48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f577 expref: 155 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091044:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d4:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.091047:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.091051:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091071:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d307b166/0xf62499ab92791c48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f577 expref: 155 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091081:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d307b166/0xf62499ab92791c48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f577 expref: 155 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.091112:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0000_UUID:14162:1871938510734720:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.091146:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd9ee600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.091164:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284603 00000100:00100000:0.0:1785220582.091171:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1269:3655:x1871938510733824:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.091183:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29355 count 1 00010000:00010000:0.0:1785220582.091193:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091203:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8fc5767/0xf62499ab927901e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f340 expref: 186 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091214:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ab:0x0].0x0 since=21s v=(0 0) 00010000:00010000:0.0:1785220582.091217:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.091221:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091239:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8fc5767/0xf62499ab927901e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f340 expref: 186 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091249:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8fc5767/0xf62499ab927901e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f340 expref: 186 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.091258:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510734848:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.091281:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.091387:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.091409:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974144 00000100:00100000:0.0:1785220582.091413:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974144 00010000:00010000:0.0:1785220582.091436:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d6ef86ed/0xf62499ab927919f5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f53f expref: 156 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.091461:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100524 00000100:00100000:0.0:1785220582.091468:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+156:2406:x1871938519974144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.091478:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.091483:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d6ef86ed/0xf62499ab927919f5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f53f expref: 156 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091501:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d6ef86ed/0xf62499ab927919f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f53f expref: 156 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091522:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.091525:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.091530:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091540:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.091555:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.091560:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.091575:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+156:2406:x1871938519974144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.091582:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100524 00000100:00100000:3.0:1785220582.091718:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1270:3653:x1871938510731392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8930us (9301us total) trans 4295070120 rc 0/0 00000100:00100000:3.0:1785220582.091725:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284590 00000100:00100000:3.0:1785220582.091745:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284604 00000100:00100000:3.0:1785220582.091761:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+76:3655:x1871938510733184:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.091773:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29395 count 1 00010000:00010000:3.0:1785220582.091786:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.091799:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000059cfb0e0/0xf62499ab92791bed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f569 expref: 155 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.091807:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d3:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.091810:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.091813:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.091839:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000059cfb0e0/0xf62499ab92791bed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f569 expref: 155 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.091850:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000059cfb0e0/0xf62499ab92791bed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f569 expref: 155 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.091861:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510734976:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.091892:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.092260:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733696:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.092291:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.092355:0:13818:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510733696 00000100:00100000:3.0:1785220582.092359:0:13818:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510733696 00000100:00100000:3.0:1785220582.092605:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284605 00000100:00100000:3.0:1785220582.092614:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+77:3655:x1871938510733696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.092633:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29399 count 1 00010000:00010000:3.0:1785220582.092647:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.092662:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000043ce59a/0xf62499ab92791e39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a1 expref: 155 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.092669:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d7:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.092673:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.092676:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.092707:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000043ce59a/0xf62499ab92791e39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a1 expref: 155 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.092720:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000043ce59a/0xf62499ab92791e39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a1 expref: 155 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.092733:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510735104:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.092787:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.093368:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.093377:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.093387:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.093391:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.093401:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.093416:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.093422:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a54a8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.093438:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.093440:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.093442:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.093834:0:14195:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d6ef86ed/0xf62499ab927919f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f53f expref: 155 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.093848:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510731904:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.093870:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.093881:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.093887:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.093896:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.093904:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.093931:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.094031:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.094038:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.094047:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.094050:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.094062:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.094073:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.094079:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a54af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.094132:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb21880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+78:3652:x1871938510731648:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8525us (8594us total) trans 4295070699 rc 0/0 00010000:00010000:0.0:1785220582.094132:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.094133:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.094135:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.094141:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284591 00000004:00080000:1.0:1785220582.094182:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b49180/2, rc 0, transno 4295070120 00000100:00100000:1.0:1785220582.094203:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510731392:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.094283:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.094294:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510735232:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.094325:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.094419:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb20e00/2, rc 0, transno 4295070699 00000100:00100000:1.0:1785220582.094426:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510735232 00000100:00100000:1.0:1785220582.094431:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510735232 00000100:00100000:2.0:1785220582.094438:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510731648:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.094464:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284606 00000100:00100000:1.0:1785220582.094473:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1270:3655:x1871938510735232:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.094491:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29358 count 1 00010000:00010000:1.0:1785220582.094508:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.094517:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510735360:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.094522:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004f6b464c/0xf62499ab927903ba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f36a expref: 186 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.094530:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ae:0x0].0x0 since=21s v=(0 0) 00010000:00010000:1.0:1785220582.094535:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.094539:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.094540:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.094561:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004f6b464c/0xf62499ab927903ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f36a expref: 186 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.094573:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004f6b464c/0xf62499ab927903ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f36a expref: 186 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.094587:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510735488:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.094642:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba300 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.094656:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510735360 00000100:00100000:1.0:1785220582.094659:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510735360 00000100:00100000:1.0:1785220582.096420:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284607 00000100:00100000:1.0:1785220582.096440:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+78:3655:x1871938510735360:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.096468:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29400 count 1 00010000:00010000:1.0:1785220582.096498:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.096520:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000385044b6/0xf62499ab92791ecc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5af expref: 154 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.096529:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d8:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.096533:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.096538:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.096572:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000385044b6/0xf62499ab92791ecc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5af expref: 154 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.096584:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000385044b6/0xf62499ab92791ecc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5af expref: 154 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.096598:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510735616:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.096677:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.098583:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.098617:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974272 00000100:00100000:2.0:1785220582.098626:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974272 00000100:00100000:3.0:1785220582.098661:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510732544:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.098662:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000042f1236f/0xf62499ab9278ffbc lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f308 expref: 187 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.098683:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.098701:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100525 00000100:00100000:2.0:1785220582.098709:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+187:2407:x1871938519974272:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.098723:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.098728:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000042f1236f/0xf62499ab9278ffbc lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f308 expref: 187 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.098741:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000042f1236f/0xf62499ab9278ffbc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f308 expref: 187 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.098775:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.098779:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.098785:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.098795:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.098818:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.098825:0:14164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042f1236f/0xf62499ab9278ffbc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f308 expref: 187 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.098856:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+187:2407:x1871938519974272:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 148us (282us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.098864:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100525 00000100:00100000:2.0:1785220582.098901:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.098902:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510732416:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.098913:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.098922:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.098928:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.098938:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.098950:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.098963:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.099015:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.099056:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510732800:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.099068:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099110:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974400 00000100:00100000:2.0:1785220582.099114:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974400 00000100:00100000:3.0:1785220582.099136:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510732672:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.099141:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c44910bd/0xf62499ab92791b3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f55b expref: 155 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099149:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099161:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974528 00000100:00100000:2.0:1785220582.099164:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974528 00000100:00100000:3.0:1785220582.099167:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100526 00000100:00100000:3.0:1785220582.099175:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9adc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+155:2407:x1871938519974400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.099175:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000d1243ac/0xf62499ab92790144 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f332 expref: 186 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099184:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.099189:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c44910bd/0xf62499ab92791b3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f55b expref: 155 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099193:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100527 00000100:00100000:2.0:1785220582.099198:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+186:2406:x1871938519974528:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.099199:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c44910bd/0xf62499ab92791b3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f55b expref: 155 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099205:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.099208:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000d1243ac/0xf62499ab92790144 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f332 expref: 186 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099213:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c44910bd/0xf62499ab92791b3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f55b expref: 155 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099218:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000d1243ac/0xf62499ab92790144 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f332 expref: 186 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099223:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.099226:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.099229:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000d1243ac/0xf62499ab92790144 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f332 expref: 186 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099230:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099238:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.099239:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099241:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72aa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.099245:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099252:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099255:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.099265:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.099277:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9adc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+154:2407:x1871938519974400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.099277:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+185:2406:x1871938519974528:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 80us (263us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.099284:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100527 00000100:00100000:3.0:1785220582.099287:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100526 00010000:00010000:3.0:1785220582.099330:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099339:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099347:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099354:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099369:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.099377:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.099393:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974784 00000100:00100000:2.0:1785220582.099397:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974784 00010000:00010000:2.0:1785220582.099410:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b50ed4e4/0xf62499ab92791a9d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f54d expref: 154 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099431:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100528 00000100:00100000:2.0:1785220582.099437:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+154:2407:x1871938519974784:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.099444:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.099448:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b50ed4e4/0xf62499ab92791a9d lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f54d expref: 154 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099457:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b50ed4e4/0xf62499ab92791a9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f54d expref: 154 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099470:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.099473:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d1:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.099474:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a52b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+79:3652:x1871938510732288:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10832us (12484us total) trans 4295070700 rc 0/0 00010000:00010000:2.0:1785220582.099477:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.099482:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284594 00010000:00010000:2.0:1785220582.099484:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099495:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.099501:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.099506:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099511:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+154:2407:x1871938519974784:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 74us (134us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.099515:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:2.0:1785220582.099517:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100528 00010000:00010000:3.0:1785220582.099520:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099529:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099541:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099548:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a54bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.099550:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb21500/2, rc 0, transno 4295070700 00000100:00100000:2.0:1785220582.099561:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb21500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510732288:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.099566:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.099569:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099575:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.099578:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099584:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099593:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099599:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a54c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099612:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.099614:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.099616:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.099620:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510735744:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.099623:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099629:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099637:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099645:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099646:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.099663:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.099666:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.099671:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099678:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.099681:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099689:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099699:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099706:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a54b6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72aa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.099718:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.099720:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.099722:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.099735:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb23b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1272:3652:x1871938510732160:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11699us (13330us total) trans 4295070122 rc 0/0 00000004:00080000:2.0:1785220582.099740:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb20700/2, rc 0, transno 4295070122 00000100:00100000:3.0:1785220582.099742:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284592 00000100:00100000:2.0:1785220582.099750:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb20700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510732160:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.099766:0:14190:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510735744 00000100:00100000:3.0:1785220582.099769:0:14190:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510735744 00000100:00100000:3.0:1785220582.099792:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284608 00000100:00100000:3.0:1785220582.099798:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+79:3652:x1871938510735744:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.099801:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.099807:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510735872:0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.099816:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29401 count 1 00000100:00100000:2.0:1785220582.099828:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.099840:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.099854:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000031d51c7d/0xf62499ab92791f43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5bd expref: 153 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099859:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.099866:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72d9:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.099869:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.099873:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099887:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.099894:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000031d51c7d/0xf62499ab92791f43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5bd expref: 153 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099902:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974656 00010000:00010000:3.0:1785220582.099905:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000031d51c7d/0xf62499ab92791f43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5bd expref: 153 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099905:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974656 00000100:00100000:3.0:1785220582.099916:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510736000:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.099918:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000816663e0/0xf62499ab9278fec0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2fa expref: 185 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.099935:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519974912 00000100:00100000:2.0:1785220582.099938:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519974912 00010000:00010000:2.0:1785220582.099949:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a483a991/0xf62499ab9279010c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f324 expref: 186 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.099951:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896000 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.099965:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510735872 00000100:00100000:2.0:1785220582.099967:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100529 00000100:00100000:3.0:1785220582.099968:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510735872 00000100:00100000:2.0:1785220582.099973:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+186:2406:x1871938519974656:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.099980:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.099985:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000816663e0/0xf62499ab9278fec0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2fa expref: 186 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.099989:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284609 00000100:00100000:3.0:1785220582.099995:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1272:3652:x1871938510735872:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.099996:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000816663e0/0xf62499ab9278fec0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2fa expref: 186 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220582.100005:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29359 count 1 00010000:00010000:2.0:1785220582.100009:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000816663e0/0xf62499ab9278fec0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f2fa expref: 186 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.100017:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100019:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.100022:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.100026:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.100028:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008f6ad793/0xf62499ab92790469 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f378 expref: 185 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100035:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.100041:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72af:0x0].0x0 since=20s v=(0 0) 00010000:00010000:3.0:1785220582.100044:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.100047:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100050:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.100064:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008f6ad793/0xf62499ab92790469 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f378 expref: 185 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.100066:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+185:2406:x1871938519974656:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (207us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.100073:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100529 00010000:00010000:3.0:1785220582.100074:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008f6ad793/0xf62499ab92790469 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f378 expref: 185 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.100106:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100530 00000100:00100000:3.0:1785220582.100109:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0001_UUID:14259:1871938510736128:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.100113:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+184:2406:x1871938519974912:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.100122:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.100126:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a483a991/0xf62499ab9279010c lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f324 expref: 184 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.100132:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896600 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.100136:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a483a991/0xf62499ab9279010c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f324 expref: 184 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100148:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.100151:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72a9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.100154:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100162:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100173:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.100192:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+184:2406:x1871938519974912:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 79us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.100201:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100530 00010000:00010000:2.0:1785220582.100219:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100228:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100237:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100243:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.100268:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.100349:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1273:3654:x1871938510731136:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11528us (13105us total) trans 4295070123 rc 0/0 00000100:00100000:2.0:1785220582.100356:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284595 00010000:00010000:3.0:1785220582.100431:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.100437:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.100444:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.100448:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.100456:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.100456:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.100472:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.100478:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a54cb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.100483:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aca680/2, rc 0, transno 4295070123 00010000:00010000:3.0:1785220582.100492:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.100492:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510731136:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.100494:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.100495:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.100530:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510736256:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.100555:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.100579:0:14185:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510736256 00000100:00100000:0.0:1785220582.100583:0:14185:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510736256 00000100:00100000:0.0:1785220582.100609:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284610 00000100:00100000:0.0:1785220582.100616:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1273:3654:x1871938510736256:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.100629:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29360 count 1 00010000:00010000:0.0:1785220582.100642:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.100654:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d0de16ed/0xf62499ab92790557 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f386 expref: 183 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.100661:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b0:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.100663:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.100667:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.100685:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d0de16ed/0xf62499ab92790557 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f386 expref: 183 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.100693:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d0de16ed/0xf62499ab92790557 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f386 expref: 183 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.100702:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510736384:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.100729:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.100819:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb22d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1273:3655:x1871938510731264:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12595us (14366us total) trans 4295070121 rc 0/0 00000100:00100000:1.0:1785220582.100879:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284593 00010000:00010000:1.0:1785220582.100991:0:13867:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b50ed4e4/0xf62499ab92791a9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f54d expref: 153 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.101003:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91583480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:13867:1871938510732928:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.101019:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.101028:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.101035:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.101044:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.101052:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.101067:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.101073:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.101105:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975040 00000100:00100000:2.0:1785220582.101109:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975040 00000100:00100000:3.0:1785220582.101137:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510734336:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.101154:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101185:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000248a3d80/0xf62499ab92791cc6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f585 expref: 153 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.101209:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100531 00000100:00100000:2.0:1785220582.101216:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+153:2407:x1871938519975040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.101227:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.101232:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000248a3d80/0xf62499ab92791cc6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f585 expref: 153 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101242:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000248a3d80/0xf62499ab92791cc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f585 expref: 153 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101255:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.101258:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.101261:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101269:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101281:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.101299:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+153:2407:x1871938519975040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.101305:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100531 00000100:00100000:2.0:1785220582.101334:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.101367:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.101383:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975168 00000100:00100000:2.0:1785220582.101386:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975168 00010000:00010000:2.0:1785220582.101400:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dc2f7c5c/0xf62499ab92791db4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f593 expref: 153 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.101410:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975296 00000100:00100000:3.0:1785220582.101414:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975296 00000100:00100000:2.0:1785220582.101418:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100532 00000100:00100000:2.0:1785220582.101424:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+153:2406:x1871938519975168:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.101429:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003a4b923c/0xf62499ab927902c5 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f34e expref: 184 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101431:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.101435:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dc2f7c5c/0xf62499ab92791db4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f593 expref: 153 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101445:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dc2f7c5c/0xf62499ab92791db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f593 expref: 153 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.101449:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100533 00000100:00100000:3.0:1785220582.101454:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+184:2407:x1871938519975296:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.101458:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc2f7c5c/0xf62499ab92791db4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f593 expref: 153 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101461:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.101465:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003a4b923c/0xf62499ab927902c5 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f34e expref: 184 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101468:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.101471:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.101476:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101480:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003a4b923c/0xf62499ab927902c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f34e expref: 184 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101484:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101492:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.101494:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ac:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.101499:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101499:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.101506:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.101515:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+152:2406:x1871938519975168:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (180us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.101517:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.101522:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100532 00000100:00100000:3.0:1785220582.101532:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+184:2407:x1871938519975296:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (165us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.101538:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100533 00010000:00010000:2.0:1785220582.101538:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101550:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101558:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101566:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.101590:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220582.101607:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.101612:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101620:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.101624:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101631:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101647:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101654:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a54d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101672:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.101676:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101682:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.101685:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101692:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101702:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101709:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a54e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.101721:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.101722:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.101725:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.101729:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000248a3d80/0xf62499ab92791cc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f585 expref: 151 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.101740:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0000_UUID:14249:1871938510734208:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.101766:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.101775:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.101781:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.101790:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.101797:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.101810:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.101932:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+80:3652:x1871938510733440:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11498us (12330us total) trans 4295070701 rc 0/0 00000100:00100000:0.0:1785220582.101940:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284598 00000004:00080000:3.0:1785220582.101959:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91583800/2, rc 0, transno 4295070701 00010000:00010000:0.0:1785220582.101968:0:14150:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003a4b923c/0xf62499ab927902c5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f34e expref: 183 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.101969:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510733440:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.101978:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0001_UUID:14150:1871938510734464:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.101990:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.102002:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.102008:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.102017:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.102024:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.102043:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220582.102126:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.102131:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102138:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.102141:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102149:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102159:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102166:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7f992b7/0xf62499ab927a54e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102181:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.102185:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102192:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.102195:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102207:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102216:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102223:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a54ee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ac:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.102235:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.102237:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.102238:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.103079:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+81:3654:x1871938510731776:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14165us (14492us total) trans 4295070702 rc 0/0 00000100:00100000:1.0:1785220582.103111:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284596 00010000:00010000:1.0:1785220582.103295:0:14187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a483a991/0xf62499ab9279010c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f324 expref: 182 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.103311:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0001_UUID:14187:1871938510733056:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.103330:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103338:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103345:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103355:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103362:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103379:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.103395:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+1273:3652:x1871938510733952:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12782us (13665us total) trans 4295070124 rc 0/0 00000100:00100000:0.0:1785220582.103402:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284600 00000100:00100000:0.0:1785220582.103438:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510734592:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.103453:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.103476:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.103481:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.103493:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:1.0:1785220582.103494:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+1273:3655:x1871938510732032:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14133us (14462us total) trans 4295070125 rc 0/0 00010000:00010000:0.0:1785220582.103497:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.103501:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284597 00010000:00010000:0.0:1785220582.103505:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103520:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.103520:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.103527:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000069a3eba5/0xf62499ab927a54d9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72a9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103528:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.103542:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.103544:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.103545:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.103583:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fc000/2, rc 0, transno 4295070124 00000004:00080000:0.0:1785220582.103583:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb22680/2, rc 0, transno 4295070702 00000100:00100000:0.0:1785220582.103588:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb22680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510731776:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.103601:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510733952:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.103685:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.103698:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510736512:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.103732:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.103773:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510736512 00000100:00100000:1.0:1785220582.103779:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510736512 00000100:00100000:1.0:1785220582.103865:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284611 00000100:00100000:1.0:1785220582.103874:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1273:3654:x1871938510736512:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.103898:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29361 count 1 00010000:00010000:1.0:1785220582.103917:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103933:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fdfb5cc5/0xf62499ab92790630 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f394 expref: 181 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103942:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b1:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.103946:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.103951:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103978:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fdfb5cc5/0xf62499ab92790630 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f394 expref: 181 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.103990:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fdfb5cc5/0xf62499ab92790630 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f394 expref: 181 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.104003:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510736640:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.104042:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.104831:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94ac9f80/2, rc 0, transno 4295070121 00000100:00100000:0.0:1785220582.104852:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510731264:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.104873:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfb23480/2, rc 0, transno 4295070125 00000100:00100000:0.0:1785220582.104877:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfb23480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510732032:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.104939:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91583480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510736896:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.104979:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.105005:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510736896 00000100:00100000:1.0:1785220582.105009:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510736896 00000100:00100000:1.0:1785220582.105042:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284612 00000100:00100000:1.0:1785220582.105050:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1274:3653:x1871938510736896:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.105066:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29363 count 1 00010000:00010000:1.0:1785220582.105082:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.105168:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.105204:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000261533f4/0xf62499ab92790733 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b0 expref: 181 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.105210:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0000_UUID:14162:1871938510734720:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.105213:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b3:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.105217:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.105221:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.105228:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.105243:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000261533f4/0xf62499ab92790733 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b0 expref: 181 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.105253:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000261533f4/0xf62499ab92790733 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b0 expref: 181 pid: 8752 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.105265:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510737024:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.105269:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510734848:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.105281:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.105302:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.105306:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.105316:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220582.105321:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91581f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510736768:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.105339:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.105341:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.105359:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510736768 00000100:00100000:1.0:1785220582.105362:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510736768 00000100:00100000:1.0:1785220582.105394:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284613 00000100:00100000:1.0:1785220582.105401:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1275:3653:x1871938510736768:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.105413:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29362 count 1 00010000:00010000:1.0:1785220582.105424:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.105436:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000046077e79/0xf62499ab927906ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a2 expref: 181 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.105443:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b2:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.105446:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.105449:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.105468:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000046077e79/0xf62499ab927906ae lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a2 expref: 181 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.105477:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000046077e79/0xf62499ab927906ae lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a2 expref: 181 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.105488:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510737152:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.105570:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975424 00000100:00100000:3.0:1785220582.105576:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975424 00010000:00010000:3.0:1785220582.105599:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f88cc130/0xf62499ab92790358 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f35c expref: 182 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.105619:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975680 00000100:00100000:3.0:1785220582.105622:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975680 00010000:00010000:3.0:1785220582.105633:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f8fc5767/0xf62499ab927901e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f340 expref: 183 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.105649:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975552 00000100:00100000:3.0:1785220582.105651:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975552 00010000:00010000:3.0:1785220582.105664:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d307b166/0xf62499ab92791c48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f577 expref: 151 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.105679:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100534 00000100:00100000:3.0:1785220582.105685:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+183:2406:x1871938519975424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.105694:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.105699:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f88cc130/0xf62499ab92790358 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f35c expref: 183 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.105708:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f88cc130/0xf62499ab92790358 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f35c expref: 183 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.105729:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f88cc130/0xf62499ab92790358 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f35c expref: 183 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.105743:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ad:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.105745:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ad:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.106133:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100535 00000100:00100000:0.0:1785220582.106144:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+182:2406:x1871938519975680:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.106153:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.106159:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f8fc5767/0xf62499ab927901e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f340 expref: 182 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.106170:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f8fc5767/0xf62499ab927901e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f340 expref: 182 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.106184:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8fc5767/0xf62499ab927901e5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f340 expref: 182 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.106195:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.106198:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ab:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.106202:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.106208:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.106216:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.106236:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.106269:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.106280:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.106303:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.106578:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+80:3652:x1871938510733568:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16036us (16927us total) trans 4295070703 rc 0/0 00000100:00100000:2.0:1785220582.106587:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284599 00000100:00100000:2.0:1785220582.107005:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100536 00000100:00100000:2.0:1785220582.107016:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+151:2406:x1871938519975552:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.107029:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.107035:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d307b166/0xf62499ab92791c48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f577 expref: 151 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107045:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d307b166/0xf62499ab92791c48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f577 expref: 151 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107060:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d307b166/0xf62499ab92791c48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f577 expref: 151 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107077:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.107080:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.107209:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107220:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107241:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.107278:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+150:2406:x1871938519975552:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 264us (1937us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.107285:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100536 00010000:00010000:2.0:1785220582.107313:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107323:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107332:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107339:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107357:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.107519:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+80:3655:x1871938510733312:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16520us (17375us total) trans 4295070704 rc 0/0 00000100:00100000:2.0:1785220582.107526:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284602 00010000:00010000:2.0:1785220582.107913:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.107921:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107929:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.107933:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107942:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.107955:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.108017:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510737280:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.108035:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+181:2406:x1871938519975680:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1893us (2726us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.108043:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100535 00000100:00100000:1.0:1785220582.108056:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.108069:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.108077:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.108115:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.108122:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510737408:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.108146:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.109026:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+180:2406:x1871938519975424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 3334us (3853us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.109036:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100534 00000004:00080000:3.0:1785220582.109115:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91580700/2, rc 0, transno 4295070703 00000100:00100000:3.0:1785220582.109129:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91580700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510733568:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.109257:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a54fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109286:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.109288:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.109290:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.109303:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510737280 00000100:00100000:2.0:1785220582.109311:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510737280 00000100:00100000:2.0:1785220582.109353:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510737408 00000100:00100000:2.0:1785220582.109357:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510737408 00000100:00100000:2.0:1785220582.109375:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284614 00000100:00100000:2.0:1785220582.109383:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+81:3654:x1871938510737280:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.109407:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29402 count 1 00010000:00010000:2.0:1785220582.109425:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109440:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c9a7900d/0xf62499ab92791fba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5cb expref: 149 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.109440:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.109450:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72da:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.109454:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.109459:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109486:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c9a7900d/0xf62499ab92791fba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5cb expref: 149 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109498:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c9a7900d/0xf62499ab92791fba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5cb expref: 149 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.109512:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510737536:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.109549:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510735488:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.109557:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6e40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.109568:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.109604:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220582.109613:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510737664:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.109624:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510735616:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.109636:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.109640:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.109655:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510737664 00000100:00100000:1.0:1785220582.109658:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510737664 00000100:00100000:2.0:1785220582.109677:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.109680:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284615 00000100:00100000:1.0:1785220582.109686:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+82:3654:x1871938510737408:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.109692:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975808 00000100:00100000:2.0:1785220582.109696:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975808 00002000:00080000:1.0:1785220582.109699:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29403 count 1 00010000:00010000:1.0:1785220582.109710:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109712:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000059cfb0e0/0xf62499ab92791bed lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f569 expref: 150 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.109727:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000702e2707/0xf62499ab9279204d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d9 expref: 150 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.109730:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976064 00000100:00100000:2.0:1785220582.109733:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976064 00010000:00010000:1.0:1785220582.109734:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72db:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.109737:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.109741:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109745:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004f6b464c/0xf62499ab927903ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f36a expref: 180 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.109772:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000702e2707/0xf62499ab9279204d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d9 expref: 150 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.109774:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100537 00000100:00100000:2.0:1785220582.109780:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+150:2406:x1871938519975808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.109782:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000702e2707/0xf62499ab9279204d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d9 expref: 150 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109787:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.109792:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510737792:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.109793:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000059cfb0e0/0xf62499ab92791bed lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f569 expref: 150 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109803:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000059cfb0e0/0xf62499ab92791bed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f569 expref: 150 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.109813:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba0c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.109821:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.109824:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.109828:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109837:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109850:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.109856:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.109869:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+150:2406:x1871938519975808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 89us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.109876:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100537 00000100:00100000:1.0:1785220582.109885:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.109890:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519975936 00000100:00100000:2.0:1785220582.109893:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519975936 00010000:00010000:2.0:1785220582.109905:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000043ce59a/0xf62499ab92791e39 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a1 expref: 150 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.109923:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976192 00000100:00100000:2.0:1785220582.109926:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976192 00010000:00010000:2.0:1785220582.109937:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000385044b6/0xf62499ab92791ecc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5af expref: 151 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.109962:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100538 00000100:00100000:1.0:1785220582.109967:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100539 00000100:00100000:2.0:1785220582.109969:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6eca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+180:2407:x1871938519976064:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.109974:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+151:2406:x1871938519975936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.109977:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.109981:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004f6b464c/0xf62499ab927903ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f36a expref: 180 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.109982:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.109986:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000043ce59a/0xf62499ab92791e39 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a1 expref: 151 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.109990:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004f6b464c/0xf62499ab927903ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f36a expref: 180 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110001:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000043ce59a/0xf62499ab92791e39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a1 expref: 151 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.110005:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f6b464c/0xf62499ab927903ba lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f36a expref: 180 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110013:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.110016:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.110016:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.110019:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ae:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.110021:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.110022:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.110030:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110030:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110042:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.110046:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.110059:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6eca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+179:2407:x1871938519976064:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (382us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.110059:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+151:2406:x1871938519975936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.110066:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100538 00010000:00010000:3.0:1785220582.110139:0:13818:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000043ce59a/0xf62499ab92791e39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a1 expref: 151 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.110149:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510735104:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.110161:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110169:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110175:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110182:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110189:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110205:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.110336:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+83:3655:x1871938510733696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17724us (18044us total) trans 4295070705 rc 0/0 00000100:00100000:3.0:1785220582.110343:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284605 00000100:00100000:3.0:1785220582.110362:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284616 00000100:00100000:3.0:1785220582.110368:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+82:3653:x1871938510737664:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.110380:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29404 count 1 00010000:00010000:3.0:1785220582.110391:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110401:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f59bd475/0xf62499ab927920f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e7 expref: 150 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110408:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72dc:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.110411:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.110412:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110415:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110422:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.110434:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f59bd475/0xf62499ab927920f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e7 expref: 150 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110437:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220582.110443:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f59bd475/0xf62499ab927920f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e7 expref: 150 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.110454:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510737920:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.110477:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.110529:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1276:3655:x1871938510733824:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19358us (20344us total) trans 4295070126 rc 0/0 00000100:00100000:0.0:1785220582.110537:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284603 00010000:00010000:0.0:1785220582.110562:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110577:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110585:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110592:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110606:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.110677:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+1276:3652:x1871938510734080:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19807us (20608us total) trans 4295070127 rc 0/0 00000100:00100000:0.0:1785220582.110684:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284601 00000100:00100000:1.0:1785220582.110690:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100539 00000100:00100000:1.0:1785220582.110705:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100540 00000100:00100000:1.0:1785220582.110711:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+149:2406:x1871938519976192:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.110720:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.110725:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000385044b6/0xf62499ab92791ecc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5af expref: 149 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.110729:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510736384:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.110736:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000385044b6/0xf62499ab92791ecc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5af expref: 149 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110745:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110766:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000385044b6/0xf62499ab92791ecc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5af expref: 149 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110770:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220582.110775:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110777:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.110780:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.110784:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.110785:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110788:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110794:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110804:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.110812:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.110817:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110824:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f496ff4c/0xf62499ab927a5511 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110843:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.110847:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110854:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.110857:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110866:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110876:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110883:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5503 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110901:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.110904:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110909:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.110912:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110919:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110928:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110935:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a54f5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.110948:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.110950:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.110951:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.110957:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.110959:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+148:2406:x1871938519976192:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 248us (1072us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.110966:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100540 00010000:00010000:1.0:1785220582.111003:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111012:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111020:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111027:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.111027:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083ed80/2, rc 0, transno 4295070705 00000100:00100000:0.0:1785220582.111037:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733696:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.111040:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.111077:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998bbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510738048:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.111133:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.111135:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+83:3655:x1871938510735360:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14704us (16593us total) trans 4295070706 rc 0/0 00000100:00100000:1.0:1785220582.111142:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284607 00000100:00100000:0.0:1785220582.111153:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510738048 00000100:00100000:0.0:1785220582.111157:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510738048 00010000:00010000:1.0:1785220582.111158:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.111163:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111169:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.111173:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111180:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284617 00010000:00010000:1.0:1785220582.111181:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111185:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+83:3653:x1871938510738048:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.111192:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.111197:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29405 count 1 00010000:00010000:1.0:1785220582.111199:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a551f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111208:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111219:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000053801cfb/0xf62499ab92792173 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5f5 expref: 147 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111221:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.111223:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.111225:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.111226:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72dd:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.111229:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.111231:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.111232:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111250:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000053801cfb/0xf62499ab92792173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5f5 expref: 147 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.111256:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.111259:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000053801cfb/0xf62499ab92792173 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5f5 expref: 147 pid: 14251 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111270:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510738176:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.111272:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976320 00000100:00100000:1.0:1785220582.111276:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976320 00010000:00010000:1.0:1785220582.111292:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008f6ad793/0xf62499ab92790469 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f378 expref: 179 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111301:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.111312:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100541 00000100:00100000:1.0:1785220582.111319:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+179:2407:x1871938519976320:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.111326:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.111331:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008f6ad793/0xf62499ab92790469 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f378 expref: 179 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111336:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.111341:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008f6ad793/0xf62499ab92790469 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f378 expref: 179 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111353:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.111356:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72af:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.111361:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111368:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111379:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:0.0:1785220582.111387:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91580e00/2, rc 0, transno 4295070704 00000100:00100000:0.0:1785220582.111395:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91580e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733312:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.111398:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+179:2407:x1871938519976320:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 81us (142us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.111405:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100541 00000100:00100000:1.0:1785220582.111416:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976576 00000100:00100000:1.0:1785220582.111419:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976576 00010000:00010000:1.0:1785220582.111431:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d0de16ed/0xf62499ab92790557 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f386 expref: 179 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.111443:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fe680/2, rc 0, transno 4295070126 00000100:00100000:1.0:1785220582.111450:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100542 00000100:00100000:0.0:1785220582.111451:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733824:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.111456:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+179:2407:x1871938519976576:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.111456:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:1.0:1785220582.111463:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220582.111463:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510738304:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.111467:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d0de16ed/0xf62499ab92790557 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f386 expref: 179 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111476:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d0de16ed/0xf62499ab92790557 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f386 expref: 179 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111485:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.111488:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d0de16ed/0xf62499ab92790557 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f386 expref: 179 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111498:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.111501:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b0:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.111503:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510738304 00010000:00010000:1.0:1785220582.111505:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111505:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510738304 00010000:00010000:1.0:1785220582.111513:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111527:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.111528:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284618 00000100:00100000:0.0:1785220582.111533:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+84:3655:x1871938510738304:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.111545:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29406 count 1 00000100:00100000:1.0:1785220582.111546:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+178:2407:x1871938519976576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.111552:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100542 00010000:00010000:0.0:1785220582.111555:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111567:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001e706de0/0xf62499ab92792214 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f603 expref: 147 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111574:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72de:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.111577:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72de:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.111578:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510736640:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.111580:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.111589:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111597:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001e706de0/0xf62499ab92792214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f603 expref: 147 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111605:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001e706de0/0xf62499ab92792214 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f603 expref: 147 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.111620:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510738432:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.111643:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.111660:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111668:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111677:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111684:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111695:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.111785:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1276:3654:x1871938510736256:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11169us (11228us total) trans 4295070128 rc 0/0 00000100:00100000:0.0:1785220582.111791:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284610 00010000:00010000:0.0:1785220582.111806:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.111810:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111817:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.111820:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111828:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111839:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111846:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a5534 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.111865:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.111867:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.111868:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.111887:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0dc00/2, rc 0, transno 4295070128 00000100:00100000:1.0:1785220582.111896:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510736256:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.111900:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000004:00080000:0.0:1785220582.111907:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0ca80/2, rc 0, transno 4295070706 00000100:00100000:0.0:1785220582.111914:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510735360:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.111928:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510738560:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.111935:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510738816:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.111952:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.111969:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.111984:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.111990:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510738688:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.112009:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.112614:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510738560 00000100:00100000:0.0:1785220582.112620:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510738560 00000100:00100000:0.0:1785220582.112891:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510738816 00000100:00100000:0.0:1785220582.112896:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510738816 00000100:00100000:1.0:1785220582.113017:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510738688 00000100:00100000:1.0:1785220582.113022:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510738688 00000100:00100000:1.0:1785220582.113070:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284619 00000100:00100000:1.0:1785220582.113078:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1277:3654:x1871938510738560:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.113139:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29364 count 1 00010000:00010000:1.0:1785220582.113160:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.113177:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000015b4c8de/0xf62499ab927907e9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3be expref: 177 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.113184:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b4:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.113188:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.113191:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.113219:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000015b4c8de/0xf62499ab927907e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3be expref: 177 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.113231:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000015b4c8de/0xf62499ab927907e9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3be expref: 177 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.113244:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94ac8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510738944:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.113299:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.113440:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284620 00000100:00100000:2.0:1785220582.113450:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+85:3653:x1871938510738816:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.113466:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29407 count 1 00010000:00010000:2.0:1785220582.113480:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.113493:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000091daf5c7/0xf62499ab927922a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f611 expref: 147 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.113500:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72df:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.113502:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.113506:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.113533:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000091daf5c7/0xf62499ab927922a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f611 expref: 147 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.113546:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000091daf5c7/0xf62499ab927922a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f611 expref: 147 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.113557:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a6eca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510739072:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.113583:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6b40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.113814:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284621 00000100:00100000:0.0:1785220582.113825:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1277:3653:x1871938510738688:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.113840:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29365 count 1 00010000:00010000:0.0:1785220582.113854:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.113867:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000028824b65/0xf62499ab9279089f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3cc expref: 177 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.113874:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b5:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.113877:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.113882:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.113902:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000028824b65/0xf62499ab9279089f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3cc expref: 177 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.113911:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000028824b65/0xf62499ab9279089f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3cc expref: 177 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.113922:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510739200:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.113947:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9540 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.113969:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.113979:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.113993:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.114000:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.114015:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.114656:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99065f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1278:3655:x1871938510735232:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20182us (20328us total) trans 4295070129 rc 0/0 00000100:00100000:0.0:1785220582.114665:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284606 00010000:00010000:0.0:1785220582.114700:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.114706:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.114716:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.114720:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.114728:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.114746:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.114765:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5518 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ae:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.114786:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.114788:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.114790:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.115322:0:14165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059cfb0e0/0xf62499ab92791bed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f569 expref: 147 pid: 14174 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.115337:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510734976:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.115361:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115370:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115377:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115387:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115396:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115422:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.115547:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+86:3655:x1871938510733184:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23795us (25312us total) trans 4295070707 rc 0/0 00000100:00100000:2.0:1785220582.115555:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284604 00000004:00080000:2.0:1785220582.115602:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fdf80/2, rc 0, transno 4295070127 00000100:00100000:2.0:1785220582.115615:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510734080:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.115678:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510739328:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.115708:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.115732:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510739328 00000100:00100000:2.0:1785220582.115737:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510739328 00000100:00100000:2.0:1785220582.115787:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284622 00000100:00100000:2.0:1785220582.115796:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1278:3654:x1871938510739328:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.115817:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29366 count 1 00010000:00010000:2.0:1785220582.115835:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115850:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a75e22cc/0xf62499ab92790939 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3da expref: 177 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115858:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b6:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.115862:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.115866:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115890:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a75e22cc/0xf62499ab92790939 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3da expref: 177 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.115902:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a75e22cc/0xf62499ab92790939 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3da expref: 177 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.115913:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a84380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510739456:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.115964:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.116063:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.116070:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116078:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.116082:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116112:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116128:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116134:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a550a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116159:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.116161:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.116163:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.116183:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.116202:0:14259:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008f6ad793/0xf62499ab92790469 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f378 expref: 177 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.116219:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0001_UUID:14259:1871938510736128:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.116233:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976448 00010000:00010000:3.0:1785220582.116234:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.116237:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976448 00010000:00010000:3.0:1785220582.116242:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.116249:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.116257:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.116257:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000031d51c7d/0xf62499ab92791f43 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5bd expref: 147 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.116265:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.116280:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100543 00010000:00010000:3.0:1785220582.116283:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.116286:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+147:2406:x1871938519976448:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.116294:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.116299:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000031d51c7d/0xf62499ab92791f43 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5bd expref: 147 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.116309:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000031d51c7d/0xf62499ab92791f43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5bd expref: 147 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.116324:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.116327:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.116332:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.116345:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.116357:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.116851:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.116858:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116867:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.116870:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116878:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116892:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116898:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cb25b45a/0xf62499ab927a552d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72af:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.116927:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.116929:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.116931:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.116963:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.117009:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0001_UUID:14248:1871938510737024:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.117025:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.117074:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510737152:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.117112:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.117145:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.117230:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.117929:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+1279:3652:x1871938510735872:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17929us (18095us total) trans 4295070130 rc 0/0 00000100:00100000:3.0:1785220582.117939:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284609 00010000:00010000:3.0:1785220582.117992:0:14190:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000031d51c7d/0xf62499ab92791f43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5bd expref: 147 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.118007:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510736000:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.118024:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118032:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118039:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118049:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118057:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118097:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.118234:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+86:3652:x1871938510735744:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18435us (18585us total) trans 4295070708 rc 0/0 00000100:00100000:3.0:1785220582.118241:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284608 00000100:00100000:3.0:1785220582.118266:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976704 00000100:00100000:3.0:1785220582.118272:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976704 00010000:00010000:1.0:1785220582.118282:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.118288:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118295:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fdfb5cc5/0xf62499ab92790630 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f394 expref: 177 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118296:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.118300:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118307:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.118322:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976832 00010000:00010000:1.0:1785220582.118322:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.118325:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976832 00010000:00010000:1.0:1785220582.118330:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a5526 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118337:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000261533f4/0xf62499ab92790733 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b0 expref: 178 pid: 8752 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.118355:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519976960 00010000:00010000:1.0:1785220582.118355:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.118357:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.118358:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519976960 00010000:00010000:1.0:1785220582.118359:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.118368:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000046077e79/0xf62499ab927906ae lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a2 expref: 179 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.118373:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100544 00000100:00100000:1.0:1785220582.118382:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+179:2407:x1871938519976704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.118385:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100545 00000100:00100000:3.0:1785220582.118390:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+179:2407:x1871938519976832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.118394:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.118396:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.118400:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000261533f4/0xf62499ab92790733 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b0 expref: 179 pid: 8752 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118400:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fdfb5cc5/0xf62499ab92790630 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f394 expref: 179 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118413:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000261533f4/0xf62499ab92790733 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b0 expref: 179 pid: 8752 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118413:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fdfb5cc5/0xf62499ab92790630 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f394 expref: 179 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118429:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fdfb5cc5/0xf62499ab92790630 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f394 expref: 179 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118430:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000261533f4/0xf62499ab92790733 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3b0 expref: 179 pid: 8752 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118442:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.118442:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.118446:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.118446:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.118452:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118452:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118462:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118463:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.118482:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.118482:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.118517:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+146:2406:x1871938519976448:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2229us (2332us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.118525:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100543 00000100:00100000:2.0:1785220582.118545:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100546 00000100:00100000:2.0:1785220582.118552:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+177:2406:x1871938519976960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.118563:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.118568:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000046077e79/0xf62499ab927906ae lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a2 expref: 177 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.118578:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000046077e79/0xf62499ab927906ae lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a2 expref: 177 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.118591:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046077e79/0xf62499ab927906ae lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3a2 expref: 177 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.118602:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.118606:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.118610:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.118622:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.118641:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.118672:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+176:2407:x1871938519976704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 291us (1708us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.118679:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100544 00010000:00010000:1.0:1785220582.118704:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118713:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118720:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118727:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.118742:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.119692:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94ac8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+175:2407:x1871938519976832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1295us (2541us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.119705:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100545 00000100:00100000:2.0:1785220582.119722:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+174:2406:x1871938519976960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1169us (2490us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.119728:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100546 00000004:00080000:2.0:1785220582.119786:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1edf80/2, rc 0, transno 4295070130 00000100:00100000:2.0:1785220582.119809:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510735872:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.119825:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b1ec380/2, rc 0, transno 4295070708 00000100:00100000:2.0:1785220582.119837:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510735744:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.119920:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.119934:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510739712:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.119983:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.120027:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510739584:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.120050:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.120053:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510739712 00000100:00100000:3.0:1785220582.120059:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510739712 00000100:00100000:3.0:1785220582.120152:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510739584 00000100:00100000:3.0:1785220582.120156:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510739584 00000100:00100000:3.0:1785220582.120177:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284623 00000100:00100000:3.0:1785220582.120184:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b0e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1280:3652:x1871938510739712:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.120185:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284624 00000100:00100000:2.0:1785220582.120191:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+86:3654:x1871938510739584:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.120206:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29367 count 1 00002000:00080000:2.0:1785220582.120206:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29408 count 1 00010000:00010000:3.0:1785220582.120225:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.120225:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.120243:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000add41e23/0xf62499ab927909e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e8 expref: 173 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.120243:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3ceb753/0xf62499ab9279232c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f61f expref: 145 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.120253:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b7:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.120256:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e0:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.120257:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.120259:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.120261:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.120262:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.120284:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d3ceb753/0xf62499ab9279232c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f61f expref: 145 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.120285:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000add41e23/0xf62499ab927909e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e8 expref: 173 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.120298:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000add41e23/0xf62499ab927909e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e8 expref: 173 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.120298:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d3ceb753/0xf62499ab9279232c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f61f expref: 145 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.120311:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510739840:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.120311:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a87480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510739968:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.120349:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d63c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.120455:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1280:3654:x1871938510736512:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16581us (16721us total) trans 4295070131 rc 0/0 00000100:00100000:1.0:1785220582.120463:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284611 00010000:00010000:1.0:1785220582.120489:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120498:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120506:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120518:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120534:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.120539:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.120667:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93d7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1280:3653:x1871938510736768:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15266us (15323us total) trans 4295070132 rc 0/0 00000100:00100000:1.0:1785220582.120675:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284613 00010000:00010000:1.0:1785220582.120695:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.120701:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.120703:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91581f80/2, rc 0, transno 4295070132 00010000:00010000:1.0:1785220582.120710:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:3.0:1785220582.120712:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91581f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510736768:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.120714:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120723:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120740:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.120746:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510740096:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.120747:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4476e42/0xf62499ab927a553b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120777:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.120781:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.120782:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.120787:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.120790:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120798:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120808:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120815:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5549 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120828:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.120830:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.120832:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.120842:0:14168:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510740096 00000100:00100000:1.0:1785220582.120846:0:14168:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510740096 00000100:00100000:1.0:1785220582.120881:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284625 00000100:00100000:1.0:1785220582.120888:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1280:3653:x1871938510740096:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.120902:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29368 count 1 00010000:00010000:1.0:1785220582.120915:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120927:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e3965da8/0xf62499ab92790a9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3f6 expref: 173 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120935:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b8:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.120937:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.120941:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120962:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e3965da8/0xf62499ab92790a9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3f6 expref: 173 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.120972:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e3965da8/0xf62499ab92790a9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3f6 expref: 173 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.120981:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510740224:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.121001:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.121017:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121026:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121034:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121041:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121061:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.121181:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfb65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+1281:3653:x1871938510736896:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16131us (16201us total) trans 4295070133 rc 0/0 00000100:00100000:1.0:1785220582.121188:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284612 00010000:00010000:1.0:1785220582.121206:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.121210:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121218:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.121221:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121228:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121240:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121247:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5542 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121263:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.121265:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.121266:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.121272:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.121287:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91583480/2, rc 0, transno 4295070133 00000100:00100000:3.0:1785220582.121296:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91583480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510736896:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.121339:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b1ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510737792:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.121357:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.121415:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.121433:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.121454:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.121472:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977216 00000100:00100000:1.0:1785220582.121476:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977216 00010000:00010000:1.0:1785220582.121494:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000702e2707/0xf62499ab9279204d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d9 expref: 146 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.121517:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977088 00000100:00100000:1.0:1785220582.121520:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977088 00010000:00010000:1.0:1785220582.121532:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c9a7900d/0xf62499ab92791fba lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5cb expref: 147 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.121547:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977344 00000100:00100000:1.0:1785220582.121549:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977344 00010000:00010000:1.0:1785220582.121560:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f59bd475/0xf62499ab927920f5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e7 expref: 148 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.121581:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100547 00000100:00100000:1.0:1785220582.121587:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9af850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+148:2406:x1871938519977216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.121598:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.121603:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000702e2707/0xf62499ab9279204d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d9 expref: 148 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121620:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000702e2707/0xf62499ab9279204d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d9 expref: 148 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121638:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000702e2707/0xf62499ab9279204d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d9 expref: 148 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121648:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.121652:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.121656:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121664:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121682:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.121697:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9af850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+147:2406:x1871938519977216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (282us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.121703:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100547 00000100:00100000:1.0:1785220582.121736:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100548 00000100:00100000:1.0:1785220582.121742:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b1ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+146:2406:x1871938519977088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.121749:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.121767:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c9a7900d/0xf62499ab92791fba lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5cb expref: 146 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121783:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c9a7900d/0xf62499ab92791fba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5cb expref: 146 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121795:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.121798:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.121802:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121810:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121822:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.121835:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+146:2406:x1871938519977088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.121841:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100548 00000100:00100000:1.0:1785220582.121854:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100549 00000100:00100000:1.0:1785220582.121860:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+145:2406:x1871938519977344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.121867:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.121871:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f59bd475/0xf62499ab927920f5 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e7 expref: 145 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121879:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f59bd475/0xf62499ab927920f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e7 expref: 145 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121894:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.121897:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.121901:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121907:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121918:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.121931:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+145:2406:x1871938519977344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 71us (476us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.121937:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100549 00010000:00010000:1.0:1785220582.121955:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121963:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121971:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121978:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.121991:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.122124:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+87:3654:x1871938510737408:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12437us (13976us total) trans 4295070709 rc 0/0 00000100:00100000:1.0:1785220582.122131:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284615 00010000:00010000:3.0:1785220582.122143:0:13818:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f59bd475/0xf62499ab927920f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e7 expref: 144 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122150:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1785220582.122155:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510737920:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.122155:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122162:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.122166:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.122169:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122174:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.122178:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.122184:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122185:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.122194:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122194:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5557 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.122202:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122208:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.122210:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.122211:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.122218:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.122220:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.122223:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122229:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.122232:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122240:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122249:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122256:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a555e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.122269:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.122270:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.122272:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.122294:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+87:3653:x1871938510737664:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11925us (12652us total) trans 4295070710 rc 0/0 00000100:00100000:3.0:1785220582.122300:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284616 00000004:00080000:3.0:1785220582.122330:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ef800/2, rc 0, transno 4295070710 00000100:00100000:3.0:1785220582.122339:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510737664:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.122891:0:14188:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9a7900d/0xf62499ab92791fba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5cb expref: 143 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.122906:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ec700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510737536:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.122921:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.122929:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.122939:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.122947:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.122954:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.122973:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.123069:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+87:3654:x1871938510737280:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13685us (15010us total) trans 4295070711 rc 0/0 00000100:00100000:2.0:1785220582.123077:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284614 00000100:00100000:2.0:1785220582.123182:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510740352:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.123216:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.123256:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ed880/2, rc 0, transno 4295070709 00000100:00100000:2.0:1785220582.123266:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510737408:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.123297:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510740608:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.123407:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.123433:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510740352 00000100:00100000:2.0:1785220582.123438:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510740352 00000100:00100000:2.0:1785220582.123469:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510740608 00000100:00100000:2.0:1785220582.123472:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510740608 00000100:00100000:2.0:1785220582.123488:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284626 00000100:00100000:2.0:1785220582.123494:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1281:3654:x1871938510740352:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.123513:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29369 count 1 00010000:00010000:2.0:1785220582.123531:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123545:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bf04fad4/0xf62499ab92790b2a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f404 expref: 173 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123553:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72b9:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.123556:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.123561:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123585:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bf04fad4/0xf62499ab92790b2a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f404 expref: 173 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123595:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bf04fad4/0xf62499ab92790b2a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f404 expref: 173 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.123607:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510740736:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.123642:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6c00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.123659:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284627 00000100:00100000:2.0:1785220582.123666:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+87:3652:x1871938510740608:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.123679:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29410 count 1 00010000:00010000:2.0:1785220582.123697:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123716:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002625c247/0xf62499ab927924d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f63b expref: 142 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123722:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e2:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.123725:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.123729:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123748:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002625c247/0xf62499ab927924d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f63b expref: 142 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.123767:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002625c247/0xf62499ab927924d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f63b expref: 142 pid: 14232 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.123777:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0000_UUID:14200:1871938510740864:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.123794:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6540 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.123824:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a6ee680/2, rc 0, transno 4295070711 00000100:00100000:2.0:1785220582.123832:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510737280:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.123862:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.123868:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d967c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510740992:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.123899:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.123931:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fc700/2, rc 0, transno 4295070131 00000100:00100000:2.0:1785220582.123939:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510736512:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.123947:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.123952:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98a85f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510740480:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.123969:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.124288:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.124292:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510740992 00000100:00100000:3.0:1785220582.124296:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510740992 00010000:00010000:1.0:1785220582.124311:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.124317:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124326:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.124330:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124337:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124358:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124365:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5550 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124381:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.124383:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.124385:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.124395:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510740480 00000100:00100000:1.0:1785220582.124398:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510740480 00000100:00100000:1.0:1785220582.124423:0:13869:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284628 00000100:00100000:1.0:1785220582.124431:0:13869:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+89:3654:x1871938510740480:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.124444:0:13869:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29409 count 1 00010000:00010000:1.0:1785220582.124457:0:13869:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124468:0:13869:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e7fec872/0xf62499ab927923e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f62d expref: 142 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124476:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e1:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.124479:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.124483:0:13869:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124504:0:13869:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e7fec872/0xf62499ab927923e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f62d expref: 142 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.124514:0:13869:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e7fec872/0xf62499ab927923e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f62d expref: 142 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.124524:0:13869:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0000_UUID:13869:1871938510741120:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.124552:0:13869:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.124621:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977472 00000100:00100000:2.0:1785220582.124625:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977472 00010000:00010000:2.0:1785220582.124647:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000053801cfb/0xf62499ab92792173 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5f5 expref: 143 pid: 14251 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.124675:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100550 00000100:00100000:2.0:1785220582.124682:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+143:2406:x1871938519977472:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.124690:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.124695:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000053801cfb/0xf62499ab92792173 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5f5 expref: 143 pid: 14251 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.124706:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000053801cfb/0xf62499ab92792173 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5f5 expref: 143 pid: 14251 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.124707:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284629 00000100:00100000:3.0:1785220582.124714:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+89:3652:x1871938510740992:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.124721:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.124724:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72dd:0x0].0x0 since=0s v=(1 0) 00002000:00080000:3.0:1785220582.124725:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29411 count 1 00010000:00010000:2.0:1785220582.124730:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.124736:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.124738:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.124747:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000086b9d344/0xf62499ab9279259b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f649 expref: 143 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.124762:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.124766:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e3:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.124769:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.124772:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.124779:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+143:2406:x1871938519977472:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (492us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.124789:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100550 00010000:00010000:3.0:1785220582.124797:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000086b9d344/0xf62499ab9279259b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f649 expref: 143 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.124806:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000086b9d344/0xf62499ab9279259b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f649 expref: 142 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.124816:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510741248:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.124896:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.125018:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510741376:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.125053:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.125154:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510741376 00000100:00100000:2.0:1785220582.125158:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510741376 00000100:00100000:2.0:1785220582.125187:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284630 00000100:00100000:2.0:1785220582.125194:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1282:3654:x1871938510741376:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.125206:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29370 count 1 00010000:00010000:2.0:1785220582.125220:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.125233:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000058d68566/0xf62499ab92790bee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f412 expref: 173 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.125239:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ba:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.125242:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.125245:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.125263:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000058d68566/0xf62499ab92790bee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f412 expref: 173 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.125272:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000058d68566/0xf62499ab92790bee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f412 expref: 173 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.125288:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510741504:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.125309:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6a80 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.125340:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99065880/2, rc 0, transno 4295070129 00000100:00100000:2.0:1785220582.125347:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510735232:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.125361:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91582a00/2, rc 0, transno 4295070707 00000100:00100000:2.0:1785220582.125371:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510733184:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.125404:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.125412:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510741632:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.125437:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.125531:0:14163:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510741632 00000100:00100000:1.0:1785220582.125536:0:14163:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510741632 00000100:00100000:1.0:1785220582.125567:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284631 00000100:00100000:1.0:1785220582.125574:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+90:3654:x1871938510741632:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.125586:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29412 count 1 00010000:00010000:1.0:1785220582.125598:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.125610:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ba032fe7/0xf62499ab9279262e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f657 expref: 142 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.125617:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e4:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.125620:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.125624:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.125643:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ba032fe7/0xf62499ab9279262e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f657 expref: 142 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.125655:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ba032fe7/0xf62499ab9279262e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f657 expref: 142 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.125665:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510741760:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.125690:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.125741:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.125986:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0000_UUID:14178:1871938510738432:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.126009:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126036:0:14156:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053801cfb/0xf62499ab92792173 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5f5 expref: 142 pid: 14251 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.126050:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510738176:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.126062:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126071:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126078:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126127:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126135:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126153:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.126310:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+91:3653:x1871938510738048:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15123us (15175us total) trans 4295070712 rc 0/0 00000100:00100000:0.0:1785220582.126318:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284617 00000100:00100000:0.0:1785220582.126341:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977600 00000100:00100000:0.0:1785220582.126345:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977600 00010000:00010000:0.0:1785220582.126362:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001e706de0/0xf62499ab92792214 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f603 expref: 142 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.126385:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100551 00000100:00100000:0.0:1785220582.126391:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+142:2406:x1871938519977600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.126401:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.126405:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001e706de0/0xf62499ab92792214 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f603 expref: 142 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126415:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001e706de0/0xf62499ab92792214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f603 expref: 142 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.126434:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510741888:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.126434:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e706de0/0xf62499ab92792214 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f603 expref: 142 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126446:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.126449:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.126454:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.126462:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.126462:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126478:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.126493:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+141:2406:x1871938519977600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (752us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.126499:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100551 00000004:00080000:3.0:1785220582.126500:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d998bbb80/2, rc 0, transno 4295070712 00000100:00100000:3.0:1785220582.126510:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998bbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510738048:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.126514:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126521:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126529:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.126537:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:0.0:1785220582.126537:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.126543:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510742016:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.126548:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.126563:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.126627:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+91:3655:x1871938510738304:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15093us (15140us total) trans 4295070713 rc 0/0 00000100:00100000:0.0:1785220582.126634:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284618 00000100:00100000:0.0:1785220582.126647:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510741888 00000100:00100000:0.0:1785220582.126650:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510741888 00000100:00100000:0.0:1785220582.126671:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510742016 00000100:00100000:0.0:1785220582.126674:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510742016 00000100:00100000:0.0:1785220582.126692:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284632 00000100:00100000:0.0:1785220582.126698:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1283:3652:x1871938510741888:12345-0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.126700:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d998b9180/2, rc 0, transno 4295070713 00000100:00100000:3.0:1785220582.126708:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510738304:0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.126713:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29371 count 1 00010000:00010000:0.0:1785220582.126727:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126739:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004aa7d0e1/0xf62499ab92790c6c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f420 expref: 173 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.126742:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510742144:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.126747:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72bb:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.126749:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.126766:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.126778:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.126787:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004aa7d0e1/0xf62499ab92790c6c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f420 expref: 173 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126797:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004aa7d0e1/0xf62499ab92790c6c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f420 expref: 173 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.126809:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94912300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510742272:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.126830:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9900 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.126843:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510742144 00000100:00100000:0.0:1785220582.126847:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510742144 00000100:00100000:0.0:1785220582.126880:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284633 00000100:00100000:0.0:1785220582.126887:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+92:3652:x1871938510742016:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.126899:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29413 count 1 00000100:00100000:3.0:1785220582.126903:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284634 00000100:00100000:3.0:1785220582.126911:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+92:3654:x1871938510742144:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.126911:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220582.126921:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29414 count 1 00010000:00010000:0.0:1785220582.126922:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007e1c2d10/0xf62499ab927926dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f665 expref: 140 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126929:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e5:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.126932:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126932:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.126936:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.126943:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007ab65a74/0xf62499ab92792746 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f673 expref: 140 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.126949:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e6:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.126951:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.126953:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007e1c2d10/0xf62499ab927926dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f665 expref: 140 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.126955:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.126963:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007e1c2d10/0xf62499ab927926dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f665 expref: 140 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.126971:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007ab65a74/0xf62499ab92792746 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f673 expref: 140 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.126973:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94911500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510742400:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.126980:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007ab65a74/0xf62499ab92792746 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f673 expref: 140 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.126989:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510742528:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.126991:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.127004:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.127406:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.127414:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127423:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.127428:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127436:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127449:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127456:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a5565 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127475:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.127479:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127486:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.127489:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127496:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127506:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127513:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a556c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.127527:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.127529:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.127530:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.127536:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.127561:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94ac8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510738944:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.127576:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.130309:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960e8601f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510740224:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.130345:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.130457:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510739200:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.130477:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.130493:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a6eca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510739072:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.130506:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.130534:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510739968:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.130553:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.130657:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a84380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510739456:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.130670:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.130690:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510739840:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.130701:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.131827:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.131864:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.131961:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.132080:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.132233:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977856 00000100:00100000:2.0:1785220582.132242:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977856 00010000:00010000:2.0:1785220582.132282:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000028824b65/0xf62499ab9279089f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3cc expref: 174 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.132371:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519978112 00000100:00100000:2.0:1785220582.132375:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519978112 00010000:00010000:2.0:1785220582.132391:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ceb753/0xf62499ab9279232c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f61f expref: 141 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.132409:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519978368 00000100:00100000:2.0:1785220582.132412:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519978368 00010000:00010000:2.0:1785220582.132423:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e3965da8/0xf62499ab92790a9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3f6 expref: 175 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.132445:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100552 00000100:00100000:2.0:1785220582.132454:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+175:2407:x1871938519977856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.132469:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.132475:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000028824b65/0xf62499ab9279089f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3cc expref: 175 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.132487:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000028824b65/0xf62499ab9279089f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3cc expref: 175 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.132510:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028824b65/0xf62499ab9279089f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3cc expref: 175 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.132526:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.132531:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.132536:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.132546:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.133937:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.134159:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+174:2407:x1871938519977856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1700us (2332us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.134170:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100552 00000100:00100000:2.0:1785220582.134205:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100553 00000100:00100000:2.0:1785220582.134216:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+141:2407:x1871938519978112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.134230:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.134238:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d3ceb753/0xf62499ab9279232c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f61f expref: 141 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134251:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d3ceb753/0xf62499ab9279232c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f61f expref: 141 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134274:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3ceb753/0xf62499ab9279232c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f61f expref: 141 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134289:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.134293:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.134298:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134308:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134333:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.134455:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+140:2407:x1871938519978112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 216us (2564us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.134464:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100553 00000100:00100000:2.0:1785220582.134555:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100554 00000100:00100000:2.0:1785220582.134567:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+173:2407:x1871938519978368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.134607:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.134616:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e3965da8/0xf62499ab92790a9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3f6 expref: 173 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134626:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e3965da8/0xf62499ab92790a9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3f6 expref: 173 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134640:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e3965da8/0xf62499ab92790a9e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3f6 expref: 173 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134650:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.134653:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.134657:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134665:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.134742:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.134771:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.134786:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.134797:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.134823:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+172:2407:x1871938519978368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 258us (2859us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.134842:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.134862:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.135307:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1284:3653:x1871938510738688:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21483us (23294us total) trans 4295070134 rc 0/0 00000100:00100000:0.0:1785220582.135315:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284621 00010000:00010000:0.0:1785220582.135340:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.135346:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.135354:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.135359:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.135367:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.135380:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.135387:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a5581 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.135405:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.135407:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.135408:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.135477:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.135510:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.135602:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.136297:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977728 00000100:00100000:1.0:1785220582.136297:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519977984 00000100:00100000:3.0:1785220582.136306:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977728 00000100:00100000:1.0:1785220582.136306:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519977984 00010000:00010000:3.0:1785220582.136335:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000015b4c8de/0xf62499ab927907e9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3be expref: 174 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.136335:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a75e22cc/0xf62499ab92790939 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3da expref: 174 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.136362:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519978240 00000100:00100000:1.0:1785220582.136365:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519978240 00010000:00010000:1.0:1785220582.136377:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000add41e23/0xf62499ab927909e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e8 expref: 175 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.136626:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.136691:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510741248:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.136716:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.136763:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fca80/2, rc 0, transno 4295070134 00000100:00100000:3.0:1785220582.136776:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510738688:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.136960:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100554 00000100:00100000:3.0:1785220582.137326:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519978496 00000100:00100000:3.0:1785220582.137331:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519978496 00010000:00010000:3.0:1785220582.137349:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000091daf5c7/0xf62499ab927922a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f611 expref: 140 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.137367:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519978752 00000100:00100000:3.0:1785220582.137369:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519978752 00010000:00010000:3.0:1785220582.137380:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002625c247/0xf62499ab927924d7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f63b expref: 141 pid: 14232 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.137392:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100555 00000100:00100000:3.0:1785220582.137399:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94911f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+174:2407:x1871938519977984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.137411:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.137417:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a75e22cc/0xf62499ab92790939 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3da expref: 174 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137427:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a75e22cc/0xf62499ab92790939 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3da expref: 174 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137445:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a75e22cc/0xf62499ab92790939 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3da expref: 174 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137459:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.137463:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.137468:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137477:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137495:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.137509:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137516:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137525:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137537:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137551:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.137718:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1284:3654:x1871938510739328:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21923us (22009us total) trans 4295070135 rc 0/0 00000100:00100000:3.0:1785220582.137725:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284622 00000100:00100000:3.0:1785220582.137776:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94911f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+173:2407:x1871938519977984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 376us (2299us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.137781:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100555 00000100:00100000:3.0:1785220582.137795:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100556 00000100:00100000:3.0:1785220582.137801:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+172:2407:x1871938519977728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.137809:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.137814:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000015b4c8de/0xf62499ab927907e9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3be expref: 172 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137820:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000015b4c8de/0xf62499ab927907e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3be expref: 172 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137833:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000015b4c8de/0xf62499ab927907e9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3be expref: 172 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137842:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.137845:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.137849:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137857:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137875:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.137890:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+171:2407:x1871938519977728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (5812us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.137894:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100556 00000100:00100000:3.0:1785220582.137908:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100557 00000100:00100000:3.0:1785220582.137912:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+170:2407:x1871938519978240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.137917:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.137920:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000add41e23/0xf62499ab927909e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e8 expref: 170 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137927:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000add41e23/0xf62499ab927909e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e8 expref: 170 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137935:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000add41e23/0xf62499ab927909e1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f3e8 expref: 170 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137941:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.137943:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.137946:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137954:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137965:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.137973:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137978:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137984:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137987:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.137995:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.138052:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b0e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1284:3652:x1871938510739712:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17869us (18070us total) trans 4295070136 rc 0/0 00000100:00100000:3.0:1785220582.138056:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284623 00000004:00080000:3.0:1785220582.138082:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ac700/2, rc 0, transno 4295070136 00000100:00100000:3.0:1785220582.138182:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510739712:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.138237:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510742784:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.138270:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.138313:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94910000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+169:2407:x1871938519978240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 400us (2802us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.138319:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100557 00000100:00100000:3.0:1785220582.138330:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100558 00000100:00100000:3.0:1785220582.138336:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+141:2407:x1871938519978496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.138344:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.138348:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000091daf5c7/0xf62499ab927922a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f611 expref: 141 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138357:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000091daf5c7/0xf62499ab927922a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f611 expref: 141 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138371:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091daf5c7/0xf62499ab927922a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f611 expref: 141 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138381:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.138384:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.138387:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138394:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138409:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.138423:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94913800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+140:2407:x1871938519978496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (2822us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.138432:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100558 00000100:00100000:3.0:1785220582.138442:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100559 00000100:00100000:3.0:1785220582.138447:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+139:2406:x1871938519978752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.138452:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.138457:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002625c247/0xf62499ab927924d7 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f63b expref: 139 pid: 14232 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138466:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002625c247/0xf62499ab927924d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f63b expref: 139 pid: 14232 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138476:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.138478:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.138482:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138488:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.138497:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.138508:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+139:2406:x1871938519978752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 62us (1884us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.138514:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100559 00000004:00080000:3.0:1785220582.138543:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a87800/2, rc 0, transno 4295070135 00000100:00100000:3.0:1785220582.138550:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a87800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510739328:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.138575:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220582.138581:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97840380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510742912:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.138601:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.138630:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220582.138638:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510742656:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.138659:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.138909:0:13869:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0000_UUID:13869:1871938510741120:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.138919:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.138928:0:13869:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.138949:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979008 00000100:00100000:1.0:1785220582.138953:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979008 00010000:00010000:1.0:1785220582.138970:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000086b9d344/0xf62499ab9279259b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f649 expref: 139 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.138978:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.139012:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.139213:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519978624 00000100:00100000:1.0:1785220582.139218:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519978624 00010000:00010000:1.0:1785220582.139233:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bf04fad4/0xf62499ab92790b2a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f404 expref: 169 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.139257:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519978880 00000100:00100000:1.0:1785220582.139260:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519978880 00000100:00100000:0.0:1785220582.139262:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100560 00000100:00100000:0.0:1785220582.139270:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+140:2406:x1871938519979008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.139272:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e7fec872/0xf62499ab927923e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f62d expref: 140 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139278:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.139283:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000086b9d344/0xf62499ab9279259b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f649 expref: 140 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.139288:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100561 00000100:00100000:1.0:1785220582.139294:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e12050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+169:2406:x1871938519978624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.139300:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.139300:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000086b9d344/0xf62499ab9279259b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f649 expref: 140 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139304:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bf04fad4/0xf62499ab92790b2a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f404 expref: 169 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139312:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bf04fad4/0xf62499ab92790b2a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f404 expref: 169 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139315:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086b9d344/0xf62499ab9279259b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f649 expref: 140 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139325:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.139325:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.139329:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72b9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.139329:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.139335:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139335:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139344:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139344:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139357:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.139364:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.139378:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e12050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+169:2406:x1871938519978624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.139378:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+139:2406:x1871938519979008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (458us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.139385:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100561 00000100:00100000:0.0:1785220582.139428:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100560 00000100:00100000:1.0:1785220582.139432:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100562 00000100:00100000:1.0:1785220582.139439:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+138:2406:x1871938519978880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.139446:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.139451:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e7fec872/0xf62499ab927923e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f62d expref: 138 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139461:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e7fec872/0xf62499ab927923e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f62d expref: 138 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139467:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139473:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fec872/0xf62499ab927923e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f62d expref: 138 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139476:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139484:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.139485:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139487:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.139491:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139494:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139500:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139508:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220582.139514:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.139532:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+137:2406:x1871938519978880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.139539:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100562 00010000:00010000:1.0:1785220582.139556:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.139562:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139575:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.139579:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139587:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139598:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139604:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5588 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.139638:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+93:3652:x1871938510740992:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14925us (15737us total) trans 4295070714 rc 0/0 00000100:00100000:0.0:1785220582.139645:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284629 00000100:00100000:0.0:1785220582.139659:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510742784 00000100:00100000:0.0:1785220582.139663:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510742784 00000100:00100000:0.0:1785220582.139683:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510742912 00000100:00100000:0.0:1785220582.139686:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510742912 00010000:00010000:1.0:1785220582.139696:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:0.0:1785220582.139700:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510742656 00010000:00010000:1.0:1785220582.139701:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.139702:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510742656 00010000:00010000:1.0:1785220582.139708:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.139712:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.139716:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284635 00010000:00010000:1.0:1785220582.139720:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.139721:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1286:3652:x1871938510742784:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.139730:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139737:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5596 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.139741:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29373 count 1 00010000:00010000:1.0:1785220582.139762:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.139766:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139771:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139773:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.139776:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139783:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139784:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f757c923/0xf62499ab92790dd1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f43c expref: 168 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139792:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72bd:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.139794:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139795:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.139799:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139802:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b84622b7/0xf62499ab927a55b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139817:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.139818:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.139820:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.139829:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f757c923/0xf62499ab92790dd1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f43c expref: 168 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139841:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f757c923/0xf62499ab92790dd1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f43c expref: 168 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.139852:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510743040:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.139893:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.139894:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139907:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.139908:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284636 00000100:00100000:0.0:1785220582.139916:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1286:3652:x1871938510742912:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.139917:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.139924:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.139928:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29374 count 1 00010000:00010000:1.0:1785220582.139938:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.139939:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139950:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000828718e9/0xf62499ab92790e5d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f44a expref: 168 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139956:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72be:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.139959:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.139962:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139979:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000828718e9/0xf62499ab92790e5d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f44a expref: 168 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.139994:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000828718e9/0xf62499ab92790e5d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f44a expref: 168 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.140003:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510743168:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.140024:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.140029:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b1eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+1287:3654:x1871938510738560:12345-0@lo:6:osp-syn-1-0.0 Request processed in 26953us (28076us total) trans 4295070137 rc 0/0 00000100:00100000:1.0:1785220582.140036:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284619 00000100:00100000:0.0:1785220582.140040:0:14173:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284637 00000100:00100000:0.0:1785220582.140047:0:14173:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1286:3654:x1871938510742656:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.140052:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.140057:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.140058:0:14173:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29372 count 1 00010000:00010000:1.0:1785220582.140064:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.140068:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.140068:0:14173:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140076:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.140080:0:14173:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd1f9ca3/0xf62499ab92790d5a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f42e expref: 168 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140274:0:14200:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002625c247/0xf62499ab927924d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f63b expref: 136 pid: 14232 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.140286:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0000_UUID:14200:1871938510740864:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.140302:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140309:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140316:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140324:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140331:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140343:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140345:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.140352:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5573 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140378:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.140382:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140389:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.140392:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140399:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140409:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140416:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a55ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.140419:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d967c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+93:3652:x1871938510740608:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16754us (17012us total) trans 4295070715 rc 0/0 00000100:00100000:2.0:1785220582.140426:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284627 00010000:00010000:1.0:1785220582.140429:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.140430:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.140432:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.140451:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0000_UUID:14163:1871938510741760:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.140455:0:14201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bf04fad4/0xf62499ab92790b2a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f404 expref: 168 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140462:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.140465:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510740736:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.140474:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140476:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.140482:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140484:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.140489:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140496:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140500:0:13869:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140503:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140508:0:13869:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140515:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.140515:0:13869:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140522:0:13869:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140532:0:13869:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.140584:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a6ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1287:3654:x1871938510740352:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17090us (17368us total) trans 4295070138 rc 0/0 00000100:00100000:1.0:1785220582.140588:0:13869:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+93:3654:x1871938510740480:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16158us (16619us total) trans 4295070716 rc 0/0 00000100:00100000:2.0:1785220582.140590:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284626 00000100:00100000:1.0:1785220582.140594:0:13869:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284628 00010000:00010000:1.0:1785220582.140606:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.140607:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.140611:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140618:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.140621:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140628:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140639:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140646:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a55b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140659:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.140661:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.140663:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.140673:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140682:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140683:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140690:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140691:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.140694:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140698:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140708:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140712:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.140719:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140726:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a55a4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140740:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.140743:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140749:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.140766:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140773:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140783:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140790:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a557a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.140798:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+93:3653:x1871938510738816:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27351us (28828us total) trans 4295070717 rc 0/0 00000100:00100000:2.0:1785220582.140804:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284620 00010000:00010000:1.0:1785220582.140807:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.140809:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.140811:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.140816:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.140819:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.140889:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140894:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140898:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140902:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140906:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140910:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140913:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.140917:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.140927:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1785220582.140994:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c2300/2, rc 0, transno 4295070714 00000100:00100000:1.0:1785220582.140995:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+1287:3653:x1871938510740096:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20107us (20212us total) trans 4295070139 rc 0/0 00000100:00100000:1.0:1785220582.141000:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284625 00000100:00100000:3.0:1785220582.141006:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510740992:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.141053:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d967c2a00/2, rc 0, transno 4295070715 00000100:00100000:3.0:1785220582.141060:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d967c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510740608:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.141234:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.141318:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72bc:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.141322:0:14173:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.141327:0:14173:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.141349:0:14173:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd1f9ca3/0xf62499ab92790d5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f42e expref: 167 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.141354:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.141360:0:14173:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd1f9ca3/0xf62499ab92790d5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f42e expref: 167 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.141378:0:14173:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510743552:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.141380:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.141396:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979136 00000100:00100000:1.0:1785220582.141400:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979136 00000100:00100000:0.0:1785220582.141413:0:14173:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.141420:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000058d68566/0xf62499ab92790bee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f412 expref: 168 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.141434:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94912300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510742272:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.141440:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979264 00000100:00100000:1.0:1785220582.141442:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979264 00010000:00010000:0.0:1785220582.141447:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.141453:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ba032fe7/0xf62499ab9279262e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f657 expref: 136 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.141468:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979520 00000100:00100000:0.0:1785220582.141468:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94911500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510742400:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.141470:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979520 00010000:00010000:1.0:1785220582.141482:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007e1c2d10/0xf62499ab927926dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f665 expref: 137 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.141482:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.141499:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100563 00000100:00100000:1.0:1785220582.141506:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+168:2406:x1871938519979136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.141515:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.141520:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000058d68566/0xf62499ab92790bee lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f412 expref: 168 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.141523:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ae300/2, rc 0, transno 4295070139 00010000:00010000:1.0:1785220582.141530:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000058d68566/0xf62499ab92790bee lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f412 expref: 168 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.141534:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510740096:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.141545:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ba:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.141548:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ba:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.141551:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510742528:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.141553:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.141563:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.141566:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.141579:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.141597:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+168:2406:x1871938519979136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (365us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.141601:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510743680:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.141630:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220582.141666:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d998b9500/2, rc 0, transno 4295070717 00000100:00100000:3.0:1785220582.141674:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510738816:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.141710:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220582.141716:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97840000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510743808:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.141741:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.141768:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510743680 00000100:00100000:3.0:1785220582.141771:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510743680 00000100:00100000:3.0:1785220582.141789:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510743808 00000100:00100000:3.0:1785220582.141792:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510743808 00000100:00100000:3.0:1785220582.141805:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284638 00000100:00100000:3.0:1785220582.141810:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1287:3652:x1871938510743680:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.141823:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29375 count 1 00010000:00010000:3.0:1785220582.141838:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.141851:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000298acbb6/0xf62499ab92790ee9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f458 expref: 168 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.141858:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72bf:0x0].0x0 since=20s v=(0 0) 00010000:00010000:3.0:1785220582.141861:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.141864:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.141883:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000298acbb6/0xf62499ab92790ee9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f458 expref: 168 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.141899:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000298acbb6/0xf62499ab92790ee9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f458 expref: 168 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.141909:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510743936:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.141932:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.141946:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284639 00000100:00100000:3.0:1785220582.141952:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+93:3652:x1871938510743808:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.141962:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29417 count 1 00010000:00010000:3.0:1785220582.141973:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.141987:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006c9e9237/0xf62499ab927928ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f69d expref: 137 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.141993:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e9:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.141996:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.141999:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.142014:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006c9e9237/0xf62499ab927928ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f69d expref: 137 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.142023:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006c9e9237/0xf62499ab927928ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f69d expref: 137 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.142032:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97843b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510744064:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.142050:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.142070:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97843480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510743296:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.142116:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220582.142145:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a85f80/2, rc 0, transno 4295070716 00000100:00100000:3.0:1785220582.142153:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510740480:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.142185:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ad880/2, rc 0, transno 4295070138 00000100:00100000:3.0:1785220582.142196:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510740352:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.142226:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fd880/2, rc 0, transno 4295070137 00000100:00100000:3.0:1785220582.142235:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510738560:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.142260:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c683800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510744448:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.142278:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.142304:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c680e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510744192:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.142321:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.142335:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1785220582.142339:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97840700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510743424:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.142353:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.142363:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510744320:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.142379:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.142482:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.142573:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98a85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+94:3654:x1871938510739584:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22382us (22522us total) trans 4295070718 rc 0/0 00000100:00100000:2.0:1785220582.142578:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284624 00000100:00100000:2.0:1785220582.142592:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510743296 00000100:00100000:2.0:1785220582.142595:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510743296 00000100:00100000:2.0:1785220582.142609:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510744448 00000100:00100000:2.0:1785220582.142611:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510744448 00000100:00100000:2.0:1785220582.142623:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510744192 00000100:00100000:2.0:1785220582.142626:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510744192 00000100:00100000:2.0:1785220582.142639:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510743424 00000100:00100000:2.0:1785220582.142642:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510743424 00000100:00100000:2.0:1785220582.142655:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510744320 00000100:00100000:2.0:1785220582.142658:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510744320 00000100:00100000:2.0:1785220582.142675:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284640 00000100:00100000:2.0:1785220582.142680:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+96:3654:x1871938510743296:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.142690:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29415 count 1 00010000:00010000:2.0:1785220582.142699:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.142708:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000736ff7fb/0xf62499ab927927f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f681 expref: 137 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.142712:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e7:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.142714:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.142717:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.142729:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000736ff7fb/0xf62499ab927927f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f681 expref: 137 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.142735:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000736ff7fb/0xf62499ab927927f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f681 expref: 137 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.142745:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510744576:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.142787:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d63c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.142818:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100564 00000100:00100000:2.0:1785220582.142823:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100565 00000100:00100000:2.0:1785220582.142831:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+137:2406:x1871938519979520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.142840:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.142845:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007e1c2d10/0xf62499ab927926dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f665 expref: 137 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.142849:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100563 00010000:00010000:2.0:1785220582.142857:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007e1c2d10/0xf62499ab927926dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f665 expref: 137 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.142873:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+137:2406:x1871938519979264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.142882:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.142887:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ba032fe7/0xf62499ab9279262e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f657 expref: 137 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.142896:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98a85500/2, rc 0, transno 4295070718 00010000:00010000:0.0:1785220582.142897:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ba032fe7/0xf62499ab9279262e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f657 expref: 137 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.142904:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98a85500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510739584:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.142912:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba032fe7/0xf62499ab9279262e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f657 expref: 137 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.142930:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e4:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.142933:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:0.0:1785220582.142933:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.142938:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.142940:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c682a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510744704:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.142946:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.142964:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.142966:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.142986:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+136:2406:x1871938519979264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (1630us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.142992:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100564 00000100:00100000:0.0:1785220582.143013:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510744704 00000100:00100000:0.0:1785220582.143017:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510744704 00010000:00010000:2.0:1785220582.143031:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007e1c2d10/0xf62499ab927926dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f665 expref: 135 pid: 14165 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.143038:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284641 00010000:00010000:2.0:1785220582.143041:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.143044:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e5:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.143044:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1289:3652:x1871938510744448:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.143048:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.143056:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.143056:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29377 count 1 00010000:00010000:0.0:1785220582.143068:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.143071:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.143079:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078297f88/0xf62499ab927910a9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f474 expref: 168 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.143102:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c1:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.143106:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.143110:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143201:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.143208:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143216:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.143220:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143230:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143243:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143251:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a559d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72b8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143269:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.143272:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143279:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.143282:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143288:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284642 00010000:00010000:3.0:1785220582.143290:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143296:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+97:3654:x1871938510744192:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.143300:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143307:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a558f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.143307:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29418 count 1 00010000:00010000:1.0:1785220582.143319:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143320:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.143322:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.143324:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.143331:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b0acc17a/0xf62499ab92792937 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ab expref: 134 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143337:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ea:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.143340:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.143343:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143360:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b0acc17a/0xf62499ab92792937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ab expref: 134 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143369:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b0acc17a/0xf62499ab92792937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ab expref: 134 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143378:0:14177:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000058d68566/0xf62499ab92790bee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f412 expref: 167 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143381:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510744832:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.143388:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510741504:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.143402:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143403:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.143411:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143417:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284643 00010000:00010000:3.0:1785220582.143418:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143424:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+97:3654:x1871938510743424:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.143427:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143435:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.143435:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29416 count 1 00010000:00010000:1.0:1785220582.143445:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143449:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.143457:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f74ce5fe/0xf62499ab92792873 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f68f expref: 134 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143463:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72e8:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.143466:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.143469:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143486:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f74ce5fe/0xf62499ab92792873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f68f expref: 134 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143494:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f74ce5fe/0xf62499ab92792873 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f68f expref: 134 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143503:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0000_UUID:14183:1871938510744960:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.143523:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.143539:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284644 00000100:00100000:1.0:1785220582.143545:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+1290:3654:x1871938510744320:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.143548:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+1290:3654:x1871938510741376:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18355us (18494us total) trans 4295070140 rc 0/0 00000100:00100000:3.0:1785220582.143554:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284630 00002000:00080000:1.0:1785220582.143556:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29376 count 1 00010000:00010000:1.0:1785220582.143567:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.143568:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284645 00000100:00100000:3.0:1785220582.143574:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+97:3652:x1871938510744704:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.143578:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ab80647e/0xf62499ab92790fad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f466 expref: 166 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143584:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c0:0x0].0x0 since=20s v=(0 0) 00002000:00080000:3.0:1785220582.143585:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29419 count 1 00010000:00010000:1.0:1785220582.143587:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.143591:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143593:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143603:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000db3cfb89/0xf62499ab927929ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b9 expref: 134 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143608:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ab80647e/0xf62499ab92790fad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f466 expref: 166 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143610:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72eb:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.143613:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.143617:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143629:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ab80647e/0xf62499ab92790fad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f466 expref: 166 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.143633:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000db3cfb89/0xf62499ab927929ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b9 expref: 134 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.143641:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0001_UUID:14195:1871938510745088:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.143642:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000db3cfb89/0xf62499ab927929ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b9 expref: 134 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.143651:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c682680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510745216:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.143659:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.143664:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaa240 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.143670:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.143674:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.143683:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ad500/2, rc 0, transno 4295070140 00010000:00010000:1.0:1785220582.143687:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.143690:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.143691:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510741376:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.143698:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143709:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.143716:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a55c0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ba:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.143725:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c682d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510745344:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.143731:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.143732:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.143734:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.143739:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.143747:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.143848:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078297f88/0xf62499ab927910a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f474 expref: 166 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.143854:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+134:2406:x1871938519979520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1023us (2473us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.143859:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078297f88/0xf62499ab927910a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f474 expref: 166 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.143862:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100565 00000100:00100000:0.0:1785220582.143869:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510745472:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.143883:0:14193:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510745344 00000100:00100000:2.0:1785220582.143886:0:14193:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510745344 00000100:00100000:0.0:1785220582.143888:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.143901:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.143910:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.143915:0:14193:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284646 00010000:00010000:0.0:1785220582.143919:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.143921:0:14193:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1290:3654:x1871938510745344:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.143926:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.143931:0:14193:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29378 count 1 00010000:00010000:0.0:1785220582.143940:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.143942:0:14193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.143953:0:14193:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000093318dbb/0xf62499ab9279115f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f482 expref: 166 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.143960:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c2:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.143962:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.143966:0:14193:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.143983:0:14193:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000093318dbb/0xf62499ab9279115f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f482 expref: 166 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.143992:0:14193:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000093318dbb/0xf62499ab9279115f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f482 expref: 166 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.144001:0:14193:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0001_UUID:14193:1871938510745600:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.144020:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+98:3654:x1871938510741632:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18447us (18582us total) trans 4295070719 rc 0/0 00000100:00100000:2.0:1785220582.144022:0:14193:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.144030:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284631 00010000:00010000:0.0:1785220582.144050:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144059:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144067:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144075:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144181:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.144186:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144193:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.144197:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144203:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144214:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144220:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a55c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144235:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.144238:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144243:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.144246:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144253:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144263:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.144270:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.144271:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.144273:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.144331:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbed80/2, rc 0, transno 4295070719 00000100:00100000:3.0:1785220582.144377:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510741632:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.144441:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00000100:00100000:3.0:1785220582.144446:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510745728:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.144447:0:14149:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a55d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.144474:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.144540:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+98:3652:x1871938510742016:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17652us (17975us total) trans 4295070720 rc 0/0 00000100:00100000:0.0:1785220582.144547:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284633 00000004:00080000:3.0:1785220582.144561:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4bb80/2, rc 0, transno 4295070720 00000100:00100000:0.0:1785220582.144561:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510745728 00000100:00100000:0.0:1785220582.144564:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510745728 00000100:00100000:0.0:1785220582.144584:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284647 00000100:00100000:0.0:1785220582.144590:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+98:3655:x1871938510745728:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.144601:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510742016:0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.144602:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29420 count 1 00010000:00010000:0.0:1785220582.144624:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144637:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f651185f/0xf62499ab92792a64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c7 expref: 133 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144644:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ec:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.144647:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.144651:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144669:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f651185f/0xf62499ab92792a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c7 expref: 133 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.144671:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510745856:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.144679:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f651185f/0xf62499ab92792a64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c7 expref: 133 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.144690:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510745984:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.144696:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.144707:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.144720:0:14186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510745856 00000100:00100000:0.0:1785220582.144723:0:14186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510745856 00000100:00100000:0.0:1785220582.144745:0:14186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284648 00000100:00100000:0.0:1785220582.144761:0:14186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+99:3652:x1871938510745856:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.144773:0:14186:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29421 count 1 00010000:00010000:0.0:1785220582.144784:0:14186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144795:0:14186:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000feb847b8/0xf62499ab92792afe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6d5 expref: 133 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144803:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ed:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.144805:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.144809:0:14186:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144830:0:14186:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000feb847b8/0xf62499ab92792afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6d5 expref: 133 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.144836:0:14186:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000feb847b8/0xf62499ab92792afe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6d5 expref: 133 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.144842:0:14186:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0000_UUID:14186:1871938510746112:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.144856:0:14186:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9900 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.144863:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.144934:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.145070:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.145290:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979392 00000100:00100000:2.0:1785220582.145295:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979392 00010000:00010000:2.0:1785220582.145317:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004aa7d0e1/0xf62499ab92790c6c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f420 expref: 167 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.145377:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979648 00000100:00100000:2.0:1785220582.145381:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979648 00010000:00010000:2.0:1785220582.145395:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007ab65a74/0xf62499ab92792746 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f673 expref: 134 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.145424:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100566 00000100:00100000:3.0:1785220582.145433:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa63800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+167:2406:x1871938519979392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.145443:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.145449:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004aa7d0e1/0xf62499ab92790c6c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f420 expref: 167 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.145459:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004aa7d0e1/0xf62499ab92790c6c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f420 expref: 167 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.145475:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004aa7d0e1/0xf62499ab92790c6c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f420 expref: 167 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.145487:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.145490:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.145495:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.145504:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.145923:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.145959:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa63800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+166:2406:x1871938519979392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 527us (1025us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.145967:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100566 00000100:00100000:3.0:1785220582.145985:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100567 00000100:00100000:3.0:1785220582.145993:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+134:2406:x1871938519979648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.146002:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.146007:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007ab65a74/0xf62499ab92792746 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f673 expref: 134 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.146018:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007ab65a74/0xf62499ab92792746 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f673 expref: 134 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.146034:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ab65a74/0xf62499ab92792746 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f673 expref: 134 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.146046:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.146049:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.146054:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.146068:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.146559:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.146586:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2406:x1871938519979648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 594us (1515us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.146594:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100567 00010000:00010000:0.0:1785220582.149401:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.149422:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.149435:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.149442:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.149509:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.149566:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.149576:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.149583:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.149589:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.149799:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1291:3652:x1871938510741888:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23098us (23332us total) trans 4295070141 rc 0/0 00000100:00100000:0.0:1785220582.149808:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284632 00000100:00100000:0.0:1785220582.150131:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510743040:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.150154:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.150232:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510743168:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.150247:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.150478:0:14173:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_034:lustre-OST0001_UUID:14173:1871938510743552:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.150491:0:14173:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.150589:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.150687:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.150717:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.150735:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979776 00000100:00100000:0.0:1785220582.150741:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979776 00010000:00010000:0.0:1785220582.150784:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f757c923/0xf62499ab92790dd1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f43c expref: 166 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.150815:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980032 00000100:00100000:0.0:1785220582.150818:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980032 00000100:00100000:2.0:1785220582.150937:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510744576:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.150939:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.150947:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.150951:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.150956:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.150960:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.150973:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.150992:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.150999:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915ac000/2, rc 0, transno 4295070141 00010000:00010000:1.0:1785220582.151000:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a55ce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.151011:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510741888:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.151018:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.151021:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.151027:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.151030:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.151036:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.151046:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.151053:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.151055:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.151057:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.151064:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.151077:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510746240:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.151077:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510744832:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.151106:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.151190:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.151217:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd1f9ca3/0xf62499ab92790d5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f42e expref: 167 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.151241:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980288 00000100:00100000:0.0:1785220582.151245:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980288 00010000:00010000:0.0:1785220582.151258:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006c9e9237/0xf62499ab927928ff lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f69d expref: 133 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.151279:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100568 00000100:00100000:0.0:1785220582.151287:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+167:2406:x1871938519979776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.151299:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.151305:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f757c923/0xf62499ab92790dd1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f43c expref: 167 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151315:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f757c923/0xf62499ab92790dd1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f43c expref: 167 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151331:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f757c923/0xf62499ab92790dd1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f43c expref: 167 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151343:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.151347:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.151353:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151362:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.151365:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220582.151372:0:8440:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a55dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151382:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.151418:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+166:2406:x1871938519979776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 131us (829us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.151424:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100568 00000100:00100000:0.0:1785220582.151461:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100569 00000100:00100000:0.0:1785220582.151466:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+165:2406:x1871938519980032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.151474:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.151478:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd1f9ca3/0xf62499ab92790d5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f42e expref: 165 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151488:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd1f9ca3/0xf62499ab92790d5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f42e expref: 165 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151501:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd1f9ca3/0xf62499ab92790d5a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f42e expref: 165 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.151509:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+100:3654:x1871938510742144:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24600us (24729us total) trans 4295070721 rc 0/0 00010000:00010000:0.0:1785220582.151510:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.151513:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.151517:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.151520:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284634 00010000:00010000:0.0:1785220582.151530:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.151532:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510746240 00000100:00100000:3.0:1785220582.151535:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510746240 00010000:00010000:0.0:1785220582.151546:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.151555:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284649 00000100:00100000:3.0:1785220582.151562:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1291:3652:x1871938510746240:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.151562:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+164:2406:x1871938519980032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (951us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.151568:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100569 00000100:00100000:0.0:1785220582.151578:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100570 00002000:00080000:3.0:1785220582.151581:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29379 count 1 00000100:00100000:0.0:1785220582.151584:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2406:x1871938519980288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.151590:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.151594:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006c9e9237/0xf62499ab927928ff lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f69d expref: 133 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.151600:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151604:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006c9e9237/0xf62499ab927928ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f69d expref: 133 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.151612:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ba55dac9/0xf62499ab927911ba lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f490 expref: 163 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.151618:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c3:0x0].0x0 since=20s v=(0 0) 00010000:00010000:3.0:1785220582.151621:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.151625:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.151648:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ba55dac9/0xf62499ab927911ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f490 expref: 163 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.151660:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ba55dac9/0xf62499ab927911ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f490 expref: 163 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.151673:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510746368:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.151711:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.151715:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.151722:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151731:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151748:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220582.151765:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4a680/2, rc 0, transno 4295070721 00000100:00100000:2.0:1785220582.151776:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510742144:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.151776:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2406:x1871938519980288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 192us (1059us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.151783:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100570 00010000:00010000:0.0:1785220582.151805:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151814:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.151816:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220582.151825:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510746496:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.151834:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.151842:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.151850:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.151861:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.151871:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.151875:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.151883:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.151886:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.151894:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.151905:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.151912:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a55e3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.151947:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1292:3652:x1871938510742784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12226us (13676us total) trans 4295070142 rc 0/0 00000100:00100000:0.0:1785220582.151953:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284635 00010000:00010000:2.0:1785220582.151962:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.151964:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.151966:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.151966:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510746496 00000100:00100000:0.0:1785220582.151969:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510746496 00000004:00080000:2.0:1785220582.151985:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915afb80/2, rc 0, transno 4295070142 00000100:00100000:0.0:1785220582.151989:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284650 00000100:00100000:2.0:1785220582.151994:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510742784:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.151995:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+100:3655:x1871938510746496:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.152006:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29422 count 1 00010000:00010000:0.0:1785220582.152019:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152030:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000086405aa1/0xf62499ab92792b9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6e3 expref: 132 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152041:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ee:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.152045:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.152048:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152070:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000086405aa1/0xf62499ab92792b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6e3 expref: 132 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152082:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000086405aa1/0xf62499ab92792b9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6e3 expref: 132 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.152111:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510746752:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.152160:0:14173:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152170:0:14173:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152178:0:14173:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152191:0:14173:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152208:0:14173:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.152304:0:14173:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_034:lustre-MDT0000-mdtlov_UUID+1292:3654:x1871938510742656:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12254us (13640us total) trans 4295070143 rc 0/0 00000100:00100000:0.0:1785220582.152310:0:14173:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284637 00010000:00010000:0.0:1785220582.152332:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.152340:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152347:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.152353:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152361:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152378:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152388:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a55f1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.152414:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.152416:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.152418:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.152719:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510745472:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.152742:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.152931:0:14193:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0001_UUID:14193:1871938510745600:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.152945:0:14193:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.152968:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4aa00/2, rc 0, transno 4295070143 00000100:00100000:2.0:1785220582.152977:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510742656:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.152983:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.152986:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.153024:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510746880:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.153024:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510745984:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.153037:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.153050:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.153064:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.153072:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510746880 00000100:00100000:2.0:1785220582.153076:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510746880 00000100:00100000:2.0:1785220582.153216:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284651 00000100:00100000:2.0:1785220582.153224:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1292:3652:x1871938510746880:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.153239:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29381 count 1 00010000:00010000:2.0:1785220582.153254:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.153267:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009e8a4661/0xf62499ab927912e7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ac expref: 163 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.153274:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c5:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.153277:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.153280:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.153302:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009e8a4661/0xf62499ab927912e7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ac expref: 163 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.153314:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009e8a4661/0xf62499ab927912e7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ac expref: 163 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.153325:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510747008:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.153343:0:14186:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0000_UUID:14186:1871938510746112:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.153354:0:14186:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.153390:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.153462:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.153493:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.153509:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519979904 00000100:00100000:0.0:1785220582.153516:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519979904 00010000:00010000:0.0:1785220582.153538:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000828718e9/0xf62499ab92790e5d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f44a expref: 164 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.153586:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980160 00000100:00100000:0.0:1785220582.153589:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980160 00010000:00010000:0.0:1785220582.153791:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000298acbb6/0xf62499ab92790ee9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f458 expref: 165 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.154017:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980416 00000100:00100000:0.0:1785220582.154022:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980416 00010000:00010000:0.0:1785220582.154037:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000736ff7fb/0xf62499ab927927f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f681 expref: 133 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.154057:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980544 00000100:00100000:0.0:1785220582.154060:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980544 00010000:00010000:0.0:1785220582.154072:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b0acc17a/0xf62499ab92792937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ab expref: 134 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.154107:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980800 00000100:00100000:0.0:1785220582.154111:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980800 00010000:00010000:0.0:1785220582.154174:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ab80647e/0xf62499ab92790fad lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f466 expref: 166 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.154198:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100571 00000100:00100000:0.0:1785220582.154207:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+166:2406:x1871938519979904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.154217:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.154222:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000828718e9/0xf62499ab92790e5d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f44a expref: 166 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154232:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000828718e9/0xf62499ab92790e5d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f44a expref: 166 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154253:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000828718e9/0xf62499ab92790e5d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f44a expref: 166 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154266:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.154269:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72be:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.154273:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154281:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154301:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.154327:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+165:2406:x1871938519979904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 123us (1341us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.154334:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100571 00000100:00100000:0.0:1785220582.154350:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100572 00000100:00100000:0.0:1785220582.154356:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+164:2406:x1871938519980160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.154363:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.154367:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000298acbb6/0xf62499ab92790ee9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f458 expref: 164 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154377:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000298acbb6/0xf62499ab92790ee9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f458 expref: 164 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154393:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.154396:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72bf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.154400:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154406:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154417:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.154434:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+164:2406:x1871938519980160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (1369us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.154440:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100572 00000100:00100000:0.0:1785220582.154453:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100573 00000100:00100000:0.0:1785220582.154460:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+134:2406:x1871938519980416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.154469:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.154473:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000736ff7fb/0xf62499ab927927f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f681 expref: 134 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154482:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000736ff7fb/0xf62499ab927927f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f681 expref: 134 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154494:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000736ff7fb/0xf62499ab927927f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f681 expref: 134 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154511:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.154514:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.154518:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154527:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154545:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.154561:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2406:x1871938519980416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (1174us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.154566:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100573 00000100:00100000:0.0:1785220582.154578:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100574 00000100:00100000:0.0:1785220582.154584:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+132:2406:x1871938519980544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.154591:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.154595:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b0acc17a/0xf62499ab92792937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ab expref: 132 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.154605:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b0acc17a/0xf62499ab92792937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ab expref: 132 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155315:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0acc17a/0xf62499ab92792937 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ab expref: 132 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155334:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.155337:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.155342:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155349:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155377:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.155402:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+131:2406:x1871938519980544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 818us (1939us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.155410:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100574 00000100:00100000:0.0:1785220582.155428:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100575 00000100:00100000:0.0:1785220582.155436:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ac050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+163:2406:x1871938519980800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.155445:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.155451:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ab80647e/0xf62499ab92790fad lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f466 expref: 163 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155462:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ab80647e/0xf62499ab92790fad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f466 expref: 163 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155474:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.155477:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.155482:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155490:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155501:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.155517:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ac050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+163:2406:x1871938519980800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 83us (2027us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.155523:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100575 00010000:00010000:0.0:1785220582.155572:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155581:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155590:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155598:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.155939:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.156133:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1293:3652:x1871938510742912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16217us (17529us total) trans 4295070144 rc 0/0 00000100:00100000:0.0:1785220582.156141:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284636 00000100:00100000:0.0:1785220582.156202:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.156222:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981056 00000100:00100000:0.0:1785220582.156229:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981056 00010000:00010000:0.0:1785220582.156249:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078297f88/0xf62499ab927910a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f474 expref: 163 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.156271:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100576 00000100:00100000:0.0:1785220582.156278:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+163:2406:x1871938519981056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.156286:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.156291:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078297f88/0xf62499ab927910a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f474 expref: 163 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156299:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078297f88/0xf62499ab927910a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f474 expref: 163 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156312:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078297f88/0xf62499ab927910a9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f474 expref: 163 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156323:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.156325:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.156330:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156337:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156354:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.156373:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+162:2406:x1871938519981056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (172us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.156380:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100576 00010000:00010000:0.0:1785220582.156400:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156409:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156417:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.156425:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.156441:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.156479:0:14145:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000298acbb6/0xf62499ab92790ee9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f458 expref: 161 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.156492:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510743936:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.156512:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.156557:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.156564:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.156572:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.156579:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.156650:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6c00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.156685:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.156694:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.156702:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.156710:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156812:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220582.156820:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156838:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.156843:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156852:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156864:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156871:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a55ea lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72be:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156887:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.156891:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156897:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.156900:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156906:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156915:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156922:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.156926:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156931:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.156934:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156942:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156951:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156958:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.156961:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156973:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.156976:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156983:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.156993:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157001:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.157002:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.157047:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.157121:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0000_UUID:14183:1871938510744960:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.157147:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157174:0:14195:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ab80647e/0xf62499ab92790fad lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f466 expref: 160 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.157184:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0001_UUID:14195:1871938510745088:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.157195:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157213:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157221:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157232:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157240:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157260:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.157483:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+1293:3654:x1871938510744320:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13934us (15098us total) trans 4295070145 rc 0/0 00000100:00100000:1.0:1785220582.157492:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284644 00010000:00010000:1.0:1785220582.157521:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157531:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157542:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157557:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157571:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.157768:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+101:3654:x1871938510744192:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14473us (15444us total) trans 4295070722 rc 0/0 00000100:00100000:1.0:1785220582.157778:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284642 00010000:00010000:1.0:1785220582.157798:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.157803:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157811:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.157817:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157891:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157908:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.157941:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5622 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.158108:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.158113:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.158129:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.158132:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.158139:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.158150:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.158157:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5614 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.158171:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.158173:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.158175:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.158181:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.158186:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.158960:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.158991:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.159012:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.159032:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.159051:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981312 00000100:00100000:1.0:1785220582.159058:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981312 00010000:00010000:1.0:1785220582.159110:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f651185f/0xf62499ab92792a64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c7 expref: 131 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.159144:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980672 00000100:00100000:1.0:1785220582.159148:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980672 00010000:00010000:1.0:1785220582.159160:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f74ce5fe/0xf62499ab92792873 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f68f expref: 132 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.159177:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519980928 00000100:00100000:1.0:1785220582.159179:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519980928 00010000:00010000:1.0:1785220582.159189:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000db3cfb89/0xf62499ab927929ca lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b9 expref: 133 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.159203:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981184 00000100:00100000:1.0:1785220582.159205:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981184 00010000:00010000:1.0:1785220582.159218:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000093318dbb/0xf62499ab9279115f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f482 expref: 160 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.159238:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100577 00000100:00100000:1.0:1785220582.159246:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2406:x1871938519981312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.159259:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.159265:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f651185f/0xf62499ab92792a64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c7 expref: 133 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159277:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f651185f/0xf62499ab92792a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c7 expref: 133 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159295:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f651185f/0xf62499ab92792a64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c7 expref: 133 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159309:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.159314:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.159320:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159329:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159353:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.159388:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+132:2406:x1871938519981312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 142us (430us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.159394:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100577 00000100:00100000:1.0:1785220582.159709:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100578 00000100:00100000:1.0:1785220582.159721:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+131:2406:x1871938519980672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.159729:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.159737:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f74ce5fe/0xf62499ab92792873 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f68f expref: 131 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159748:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f74ce5fe/0xf62499ab92792873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f68f expref: 131 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159773:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f74ce5fe/0xf62499ab92792873 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f68f expref: 131 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159790:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.159796:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.159800:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159808:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.159887:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.159912:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+130:2406:x1871938519980672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 193us (919us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.159922:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100578 00000100:00100000:1.0:1785220582.159938:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100579 00000100:00100000:1.0:1785220582.159944:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+129:2406:x1871938519980928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.159952:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.159971:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000db3cfb89/0xf62499ab927929ca lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b9 expref: 129 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.160028:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000db3cfb89/0xf62499ab927929ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b9 expref: 129 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.160054:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.160057:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.160061:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.160069:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.160080:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.160112:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+129:2406:x1871938519980928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 168us (1099us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.160195:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1785220582.160202:0:14202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a560d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160220:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220582.160227:0:14145:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a55f8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72bf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.160251:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100579 00000100:00100000:0.0:1785220582.160298:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100580 00000100:00100000:0.0:1785220582.160308:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+160:2406:x1871938519981184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.160317:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.160323:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000093318dbb/0xf62499ab9279115f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f482 expref: 160 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.160334:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000093318dbb/0xf62499ab9279115f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f482 expref: 160 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.160351:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093318dbb/0xf62499ab9279115f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f482 expref: 160 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.160361:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.160363:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.160367:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.160374:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.160393:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.160408:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+159:2406:x1871938519981184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (1376us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.160414:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100580 00000100:00100000:0.0:1785220582.160451:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896900 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.160452:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1293:3652:x1871938510743680:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18640us (18820us total) trans 4295070146 rc 0/0 00000100:00100000:3.0:1785220582.160459:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284638 00010000:00010000:3.0:1785220582.160502:0:14206:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c9e9237/0xf62499ab927928ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f69d expref: 128 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.160514:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97843b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0000_UUID:14206:1871938510744064:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.160534:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.160535:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1293:3652:x1871938510744448:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17491us (18255us total) trans 4295070147 rc 0/0 00000100:00100000:0.0:1785220582.160542:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284641 00010000:00010000:3.0:1785220582.160544:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160551:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160559:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160567:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160581:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.160665:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+101:3652:x1871938510743808:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18712us (18922us total) trans 4295070723 rc 0/0 00000100:00100000:3.0:1785220582.160671:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284639 00000100:00100000:0.0:1785220582.160707:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.160718:0:14177:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db3cfb89/0xf62499ab927929ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b9 expref: 127 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.160734:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c682680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0000_UUID:14177:1871938510745216:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.160747:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.160783:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510746368:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.160794:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160808:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.160808:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510746752:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.160815:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.160821:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160823:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160831:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.160844:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.160914:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+101:3652:x1871938510744704:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17339us (17948us total) trans 4295070724 rc 0/0 00000100:00100000:3.0:1785220582.160924:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284645 00000100:00100000:0.0:1785220582.160935:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.160981:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981440 00000100:00100000:1.0:1785220582.160986:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981440 00010000:00010000:2.0:1785220582.160992:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.160999:0:14174:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000492e9ee/0xf62499ab927a5606 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161003:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000feb847b8/0xf62499ab92792afe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6d5 expref: 127 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.161023:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981696 00000100:00100000:1.0:1785220582.161026:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981696 00010000:00010000:1.0:1785220582.161037:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000086405aa1/0xf62499ab92792b9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6e3 expref: 128 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.161133:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100581 00000100:00100000:1.0:1785220582.161141:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+128:2406:x1871938519981440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.161150:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.161155:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000feb847b8/0xf62499ab92792afe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6d5 expref: 128 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.161165:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+101:3654:x1871938510743296:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18405us (18968us total) trans 4295070725 rc 0/0 00010000:00010000:1.0:1785220582.161166:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000feb847b8/0xf62499ab92792afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6d5 expref: 128 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.161172:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284640 00010000:00010000:1.0:1785220582.161186:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feb847b8/0xf62499ab92792afe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6d5 expref: 128 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161196:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.161199:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.161203:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161211:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161229:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.161249:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2406:x1871938519981440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.161255:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100581 00000100:00100000:1.0:1785220582.161269:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100582 00000100:00100000:1.0:1785220582.161275:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+126:2407:x1871938519981696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.161283:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.161287:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000086405aa1/0xf62499ab92792b9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6e3 expref: 126 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161296:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000086405aa1/0xf62499ab92792b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6e3 expref: 126 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161313:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086405aa1/0xf62499ab92792b9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6e3 expref: 126 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161322:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.161324:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.161328:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161335:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.161352:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.161383:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+125:2407:x1871938519981696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (448us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.161389:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100582 00000100:00100000:0.0:1785220582.163318:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.163349:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981568 00000100:00100000:0.0:1785220582.163357:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981568 00000100:00100000:3.0:1785220582.163359:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510746624:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.163390:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ba55dac9/0xf62499ab927911ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f490 expref: 159 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.163406:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.163426:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100583 00000100:00100000:0.0:1785220582.163433:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+159:2407:x1871938519981568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.163445:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:3.0:1785220582.163450:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c681500/2, rc 0, transno 4295070145 00010000:00010000:0.0:1785220582.163450:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ba55dac9/0xf62499ab927911ba lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f490 expref: 159 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.163460:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ba55dac9/0xf62499ab927911ba lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f490 expref: 159 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.163462:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510744320:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.163474:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c680e00/2, rc 0, transno 4295070722 00000100:00100000:3.0:1785220582.163484:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c680e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510744192:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.163484:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ba55dac9/0xf62499ab927911ba lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f490 expref: 159 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.163494:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97843480/2, rc 0, transno 4295070725 00000100:00100000:3.0:1785220582.163497:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97843480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510743296:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.163500:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.163504:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.163509:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.163524:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.163555:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.163590:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2407:x1871938519981568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 157us (280us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.163598:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100583 00010000:00010000:3.0:1785220582.163624:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.163633:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.163643:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.163650:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.163663:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.163689:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510746624 00000100:00100000:0.0:1785220582.163693:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510746624 00000100:00100000:0.0:1785220582.163716:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284652 00000100:00100000:0.0:1785220582.163722:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebd8f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1293:3654:x1871938510746624:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.163745:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29380 count 1 00010000:00010000:0.0:1785220582.163779:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.163794:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bdff2947/0xf62499ab9279124d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f49e expref: 157 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.163802:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c4:0x0].0x0 since=20s v=(0 0) 00010000:00010000:0.0:1785220582.163805:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.163808:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.163821:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1294:3652:x1871938510746240:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12260us (12630us total) trans 4295070148 rc 0/0 00000100:00100000:3.0:1785220582.163831:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284649 00010000:00010000:0.0:1785220582.163919:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bdff2947/0xf62499ab9279124d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f49e expref: 157 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.163938:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bdff2947/0xf62499ab9279124d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f49e expref: 157 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.163951:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510747136:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.163982:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.163989:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.164008:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981824 00000100:00100000:0.0:1785220582.164012:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981824 00010000:00010000:0.0:1785220582.164029:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009e8a4661/0xf62499ab927912e7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ac expref: 158 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.164048:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100584 00000100:00100000:0.0:1785220582.164056:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2407:x1871938519981824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.164063:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.164067:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009e8a4661/0xf62499ab927912e7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ac expref: 158 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.164076:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009e8a4661/0xf62499ab927912e7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ac expref: 158 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.164105:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.164109:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.164114:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.164388:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.164403:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.164430:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2407:x1871938519981824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 375us (447us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.164437:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100584 00010000:00010000:1.0:1785220582.164776:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.164786:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.164795:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.164802:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.164814:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:1.0:1785220582.165990:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+101:3655:x1871938510745728:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21396us (21511us total) trans 4295070726 rc 0/0 00000100:00100000:1.0:1785220582.166000:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284647 00010000:00010000:1.0:1785220582.166032:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166042:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166052:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166060:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166070:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:1.0:1785220582.166178:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+101:3654:x1871938510743424:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22753us (23822us total) trans 4295070727 rc 0/0 00000100:00100000:1.0:1785220582.166185:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284643 00010000:00010000:1.0:1785220582.166204:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220582.166211:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166219:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.166223:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166231:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166249:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166256:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7f64e58/0xf62499ab927a55ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166277:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220582.166281:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166288:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.166290:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166297:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166305:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166312:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5629 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.166313:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97840700/2, rc 0, transno 4295070727 00000100:00100000:3.0:1785220582.166320:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97840700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510743424:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.166325:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.166329:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166335:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.166338:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166345:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166355:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166362:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f5c04ab/0xf62499ab927a5645 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166376:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.166380:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166387:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.166390:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166402:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166412:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.166416:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.166419:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a5637 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166432:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1785220582.166435:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519981952 00010000:00010000:1.0:1785220582.166435:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.166440:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519981952 00010000:00010000:1.0:1785220582.166441:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.166444:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166451:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166459:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.166465:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bdff2947/0xf62499ab9279124d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f49e expref: 158 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166466:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a561b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166486:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.166488:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.166489:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100585 00010000:00010000:1.0:1785220582.166490:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.166496:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.166497:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2406:x1871938519981952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.166507:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.166510:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.166514:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bdff2947/0xf62499ab9279124d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f49e expref: 158 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166516:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166525:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.166526:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bdff2947/0xf62499ab9279124d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f49e expref: 158 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166533:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.166545:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.166545:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.166549:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.166554:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.166563:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.166578:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.166608:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2406:x1871938519981952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (193us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.166616:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100585 00010000:00010000:2.0:1785220582.166742:0:14193:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.166763:0:14193:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.166772:0:14193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.166779:0:14193:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.166794:0:14193:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.166945:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+101:3652:x1871938510745856:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22192us (22246us total) trans 4295070728 rc 0/0 00000100:00100000:1.0:1785220582.166951:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284648 00010000:00010000:1.0:1785220582.166969:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.166972:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166979:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.166981:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.166989:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167000:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167007:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa7c81c0/0xf62499ab927a563e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167021:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.167025:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167031:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.167034:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167042:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167051:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167057:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5630 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167075:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167077:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167078:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167105:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167115:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167134:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167143:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167149:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167162:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1785220582.167286:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f725180/2, rc 0, transno 4295070726 00000100:00100000:0.0:1785220582.167292:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510745728:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.167313:0:14209:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bdff2947/0xf62499ab9279124d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f49e expref: 157 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.167324:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510747136:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.167343:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.167351:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.167358:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.167367:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.167374:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.167387:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.167585:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b4b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+101:3655:x1871938510746496:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15589us (15732us total) trans 4295070729 rc 0/0 00000100:00100000:1.0:1785220582.167592:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284650 00010000:00010000:1.0:1785220582.167608:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:2.0:1785220582.167632:0:14193:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+1295:3654:x1871938510745344:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23711us (23882us total) trans 4295070149 rc 0/0 00000100:00100000:2.0:1785220582.167640:0:14193:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284646 00010000:00010000:2.0:1785220582.167657:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.167662:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167669:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.167672:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167678:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167680:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167685:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.167689:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167697:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167698:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167706:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a565a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167707:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.167715:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a564c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167721:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.167722:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.167724:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167727:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167728:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167730:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167734:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.167742:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.167807:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebd8f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1294:3654:x1871938510746624:12345-0@lo:6:osp-syn-1-0.0 Request processed in 4084us (4400us total) trans 4295070150 rc 0/0 00000100:00100000:0.0:1785220582.167814:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284652 00010000:00010000:2.0:1785220582.167832:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e8a4661/0xf62499ab927912e7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ac expref: 156 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.167842:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510747008:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.167854:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167862:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167869:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167878:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.167889:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.167900:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b49500/2, rc 0, transno 4295070729 00010000:00010000:2.0:1785220582.167909:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.167910:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510746496:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.167923:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510747520:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.167957:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.167982:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1294:3652:x1871938510746880:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14759us (14931us total) trans 4295070151 rc 0/0 00000100:00100000:0.0:1785220582.167984:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510747520 00000100:00100000:2.0:1785220582.167988:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284651 00000100:00100000:0.0:1785220582.167988:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510747520 00000100:00100000:0.0:1785220582.168018:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284653 00000100:00100000:0.0:1785220582.168025:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+101:3655:x1871938510747520:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.168045:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29425 count 1 00010000:00010000:0.0:1785220582.168063:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.168077:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000030e1700b/0xf62499ab92792dcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f70d expref: 124 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168133:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.168137:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168145:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.168148:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168155:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168167:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168174:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000823c5b05/0xf62499ab927a5653 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168188:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.168190:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.168192:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.168282:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f1:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.168287:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.168292:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.168302:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b49880/2, rc 0, transno 4295070151 00000100:00100000:2.0:1785220582.168311:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510746880:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.168316:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000030e1700b/0xf62499ab92792dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f70d expref: 124 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.168320:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f727480/2, rc 0, transno 4295070148 00000100:00100000:2.0:1785220582.168323:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510746240:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.168329:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000030e1700b/0xf62499ab92792dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f70d expref: 124 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.168332:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724700/2, rc 0, transno 4295070728 00000100:00100000:2.0:1785220582.168337:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510745856:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.168342:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510748032:192.168.201.51@tcp:104: 00000004:00080000:2.0:1785220582.168377:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c682a00/2, rc 0, transno 4295070724 00000100:00100000:2.0:1785220582.168384:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c682a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510744704:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.168392:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.168407:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c683800/2, rc 0, transno 4295070147 00000100:00100000:0.0:1785220582.168410:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510748160:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.168413:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c683800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510744448:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.168423:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97840000/2, rc 0, transno 4295070723 00000100:00100000:2.0:1785220582.168429:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97840000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510743808:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.168434:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.168467:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97841500/2, rc 0, transno 4295070146 00000100:00100000:2.0:1785220582.168473:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510743680:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.168484:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97840380/2, rc 0, transno 4295070144 00000100:00100000:2.0:1785220582.168488:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97840380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510742912:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.168498:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510747648:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.168498:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510748672:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.168518:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.168562:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93d7ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510747264:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.168578:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.168603:0:13874:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510748160 00000100:00100000:1.0:1785220582.168608:0:13874:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510748160 00000004:00080000:3.0:1785220582.168659:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c682d80/2, rc 0, transno 4295070149 00000100:00100000:3.0:1785220582.168668:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c682d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510745344:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.168679:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510747392:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.168706:0:13874:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510748672 00000100:00100000:1.0:1785220582.168710:0:13874:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510748672 00000100:00100000:1.0:1785220582.168728:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284654 00000100:00100000:1.0:1785220582.168735:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+103:3655:x1871938510748160:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.168747:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29428 count 1 00010000:00010000:1.0:1785220582.168772:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168786:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b97d6236/0xf62499ab9279300d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f737 expref: 124 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.168794:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f4:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.168797:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.168801:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.168868:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510748288:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.168891:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.168941:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510748800:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.168961:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.169019:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.169036:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510747776:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.169055:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.169058:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510747648 00000100:00100000:0.0:1785220582.169062:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510747648 00000100:00100000:2.0:1785220582.169074:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510748288 00000100:00100000:2.0:1785220582.169077:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510748288 00010000:00010000:1.0:1785220582.169202:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b97d6236/0xf62499ab9279300d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f737 expref: 124 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169221:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b97d6236/0xf62499ab9279300d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f737 expref: 124 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.169231:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510749184:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.169272:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.169285:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510748800 00000100:00100000:1.0:1785220582.169288:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510748800 00000100:00100000:1.0:1785220582.169311:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510747264 00000100:00100000:1.0:1785220582.169314:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510747264 00000100:00100000:1.0:1785220582.169330:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510747776 00000100:00100000:1.0:1785220582.169333:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510747776 00000100:00100000:1.0:1785220582.169350:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284655 00000100:00100000:1.0:1785220582.169356:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+106:3655:x1871938510748672:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.169371:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29430 count 1 00010000:00010000:1.0:1785220582.169383:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169395:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009b7c4b22/0xf62499ab9279316b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f753 expref: 124 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169400:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f6:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.169404:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.169407:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169431:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009b7c4b22/0xf62499ab9279316b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f753 expref: 124 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169439:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009b7c4b22/0xf62499ab9279316b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f753 expref: 124 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.169448:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510749440:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.169471:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.169487:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284656 00000100:00100000:1.0:1785220582.169494:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+106:3652:x1871938510747648:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.169506:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29426 count 1 00010000:00010000:1.0:1785220582.169517:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169528:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000096c8b2ae/0xf62499ab92792e38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f71b expref: 124 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169534:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f2:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.169537:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.169540:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169558:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000096c8b2ae/0xf62499ab92792e38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f71b expref: 124 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.169572:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000096c8b2ae/0xf62499ab92792e38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f71b expref: 124 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.169581:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510749568:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.169604:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284657 00000100:00100000:1.0:1785220582.169604:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.169611:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1295:3652:x1871938510748800:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.169623:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29385 count 1 00010000:00010000:2.0:1785220582.169640:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169653:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ed3b0cd1/0xf62499ab92791621 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4e4 expref: 155 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169660:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c9:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.169662:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.169667:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169687:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ed3b0cd1/0xf62499ab92791621 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4e4 expref: 155 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169698:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ed3b0cd1/0xf62499ab92791621 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4e4 expref: 155 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.169708:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510749696:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.169731:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.169747:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284658 00000100:00100000:2.0:1785220582.169766:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+106:3653:x1871938510747264:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.169778:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29423 count 1 00010000:00010000:2.0:1785220582.169790:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169800:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000498f31a9/0xf62499ab92792c32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f1 expref: 124 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169807:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ef:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.169810:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.169813:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169831:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000498f31a9/0xf62499ab92792c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f1 expref: 124 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.169845:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000498f31a9/0xf62499ab92792c32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f1 expref: 124 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.169855:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510749824:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.169877:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.169890:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510748416:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.169915:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.170954:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510748416 00000100:00100000:2.0:1785220582.170957:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510748928:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.170960:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510748416 00000100:00100000:2.0:1785220582.170990:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.170996:0:14222:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510748928 00000100:00100000:0.0:1785220582.170999:0:14222:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510748928 00000100:00100000:0.0:1785220582.171018:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284659 00000100:00100000:2.0:1785220582.171021:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.171025:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+106:3653:x1871938510747776:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.171039:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510747904:0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.171041:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29427 count 1 00010000:00010000:0.0:1785220582.171056:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171060:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.171074:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510748544:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.171076:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a83276ce/0xf62499ab92792f26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f729 expref: 124 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171097:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f3:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.171103:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.171107:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171144:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a83276ce/0xf62499ab92792f26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f729 expref: 124 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171156:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a83276ce/0xf62499ab92792f26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f729 expref: 124 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.171166:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510749952:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.171275:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510747392 00000100:00100000:3.0:1785220582.171280:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510747392 00000100:00100000:3.0:1785220582.171306:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510747904 00000100:00100000:3.0:1785220582.171308:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510747904 00000100:00100000:0.0:1785220582.171310:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.171335:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284660 00000100:00100000:1.0:1785220582.171340:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284661 00000100:00100000:3.0:1785220582.171342:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1298:3652:x1871938510748288:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.171347:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1298:3653:x1871938510748416:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.171355:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29383 count 1 00002000:00080000:1.0:1785220582.171355:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29384 count 1 00010000:00010000:3.0:1785220582.171368:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.171368:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.171381:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002b8877a1/0xf62499ab927914a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c8 expref: 155 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.171381:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000095286827/0xf62499ab9279156b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4d6 expref: 155 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.171389:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c7:0x0].0x0 since=20s v=(0 0) 00010000:00010000:1.0:1785220582.171389:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c8:0x0].0x0 since=20s v=(0 0) 00010000:00010000:3.0:1785220582.171392:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.171392:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.171397:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.171397:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.171415:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000095286827/0xf62499ab9279156b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4d6 expref: 155 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.171417:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002b8877a1/0xf62499ab927914a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c8 expref: 155 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.171427:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002b8877a1/0xf62499ab927914a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c8 expref: 155 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.171427:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000095286827/0xf62499ab9279156b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4d6 expref: 155 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.171438:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510750208:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.171442:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510750080:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.171461:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaa240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.171466:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.171493:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284662 00000100:00100000:2.0:1785220582.171497:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.171501:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1298:3653:x1871938510748928:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.171512:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29386 count 1 00000100:00100000:2.0:1785220582.171518:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510748544 00000100:00100000:2.0:1785220582.171521:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510748544 00010000:00010000:0.0:1785220582.171524:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171536:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003bb1175e/0xf62499ab927916c2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f2 expref: 155 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171543:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284663 00000100:00100000:2.0:1785220582.171548:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+108:3654:x1871938510747392:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.171548:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ca:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.171551:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.171554:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.171558:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29424 count 1 00010000:00010000:2.0:1785220582.171568:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171572:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003bb1175e/0xf62499ab927916c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f2 expref: 155 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.171580:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000632d6a7b/0xf62499ab92792ccc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ff expref: 124 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171581:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003bb1175e/0xf62499ab927916c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f2 expref: 155 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.171588:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f0:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.171591:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f0:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.171591:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510750336:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.171595:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.171610:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000632d6a7b/0xf62499ab92792ccc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ff expref: 124 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.171619:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000632d6a7b/0xf62499ab92792ccc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ff expref: 124 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171629:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510750464:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.171648:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d66c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.171663:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284664 00000100:00100000:2.0:1785220582.171671:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1298:3654:x1871938510747904:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.171682:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29382 count 1 00000100:00100000:0.0:1785220582.171693:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.171694:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.171704:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001e70c6f2/0xf62499ab927913ce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ba expref: 155 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.171710:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284665 00010000:00010000:2.0:1785220582.171711:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72c6:0x0].0x0 since=20s v=(0 0) 00010000:00010000:2.0:1785220582.171713:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c6:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.171717:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+108:3654:x1871938510748544:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.171718:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.171728:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29429 count 1 00010000:00010000:2.0:1785220582.171734:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001e70c6f2/0xf62499ab927913ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ba expref: 155 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171740:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.171743:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001e70c6f2/0xf62499ab927913ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ba expref: 155 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171761:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007ef91bf3/0xf62499ab9279308b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f745 expref: 124 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171765:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510750592:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.171769:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f5:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.171771:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.171775:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171783:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6c00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.171793:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007ef91bf3/0xf62499ab9279308b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f745 expref: 124 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171798:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510749056:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.171802:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007ef91bf3/0xf62499ab9279308b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f745 expref: 124 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.171811:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0000_UUID:14210:1871938510750720:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.171820:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.171838:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.171852:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510749056 00000100:00100000:0.0:1785220582.171855:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510749056 00000004:00080000:2.0:1785220582.171856:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b48e00/2, rc 0, transno 4295070150 00000100:00100000:2.0:1785220582.171866:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510746624:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.171878:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284666 00000100:00100000:0.0:1785220582.171883:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1299:3654:x1871938510749056:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.171893:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29387 count 1 00010000:00010000:0.0:1785220582.171904:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171915:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510750848:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.171916:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000060d06c4f/0xf62499ab9279176a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f500 expref: 155 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.171922:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72cb:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.171925:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.171928:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171939:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.171951:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000060d06c4f/0xf62499ab9279176a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f500 expref: 155 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.171958:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510750848 00000100:00100000:2.0:1785220582.171961:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510750848 00010000:00010000:0.0:1785220582.171961:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000060d06c4f/0xf62499ab9279176a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f500 expref: 155 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.171970:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510750976:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.171982:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284667 00000100:00100000:2.0:1785220582.171988:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1300:3652:x1871938510750848:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.171989:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00002000:00080000:2.0:1785220582.171998:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29389 count 1 00010000:00010000:2.0:1785220582.172009:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172020:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001701cb7a/0xf62499ab927918a5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f51c expref: 155 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172027:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72cd:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.172030:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.172033:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.172049:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510748032:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.172050:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001701cb7a/0xf62499ab927918a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f51c expref: 155 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172058:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001701cb7a/0xf62499ab927918a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f51c expref: 155 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.172067:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510751104:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.172068:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.172295:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6540 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.172339:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.172346:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510749312:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.172374:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.172520:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.172551:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510749312 00000100:00100000:0.0:1785220582.172556:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510749312 00000100:00100000:0.0:1785220582.172581:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284668 00000100:00100000:0.0:1785220582.172586:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1301:3654:x1871938510749312:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.172598:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29388 count 1 00000100:00100000:2.0:1785220582.172650:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982080 00000100:00100000:2.0:1785220582.172654:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982080 00010000:00010000:2.0:1785220582.172674:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000030e1700b/0xf62499ab92792dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f70d expref: 125 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.172683:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.172696:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e6099ad4/0xf62499ab927917f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f50e expref: 155 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.172698:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100586 00000100:00100000:2.0:1785220582.172704:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+125:2407:x1871938519982080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.172709:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72cc:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.172712:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.172713:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.172716:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172719:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000030e1700b/0xf62499ab92792dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f70d expref: 125 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172729:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000030e1700b/0xf62499ab92792dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f70d expref: 125 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.172735:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e6099ad4/0xf62499ab927917f6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f50e expref: 155 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.172743:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e6099ad4/0xf62499ab927917f6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f50e expref: 155 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172748:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030e1700b/0xf62499ab92792dcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f70d expref: 125 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.172764:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510751232:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.172784:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.172795:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.172798:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.172802:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172811:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172834:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.172841:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.172849:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.172852:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+124:2407:x1871938519982080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 149us (333us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.172858:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.172860:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100586 00010000:00010000:0.0:1785220582.172865:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.172888:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.172898:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.172903:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172911:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.172915:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172923:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172934:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172940:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5661 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.172957:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.172958:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.172960:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.173033:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+109:3655:x1871938510747520:12345-0@lo:6:osp-syn-0-0.0 Request processed in 5006us (5074us total) trans 4295070730 rc 0/0 00000100:00100000:0.0:1785220582.173039:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284653 00000004:00080000:0.0:1785220582.173073:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72d500/2, rc 0, transno 4295070730 00000100:00100000:0.0:1785220582.173110:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510747520:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.173175:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.173183:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510751360:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.173212:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.173916:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510751360 00000100:00100000:1.0:1785220582.173924:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510751360 00000100:00100000:1.0:1785220582.173975:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284669 00000100:00100000:1.0:1785220582.173984:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b10ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+109:3652:x1871938510751360:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.174004:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29431 count 1 00010000:00010000:1.0:1785220582.174022:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.174039:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b1e786f5/0xf62499ab927931b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f761 expref: 123 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.174047:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f7:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.174051:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.174055:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.174079:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b1e786f5/0xf62499ab927931b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f761 expref: 123 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.174120:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b1e786f5/0xf62499ab927931b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f761 expref: 123 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.174136:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510751488:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.174182:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.178743:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510749184:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.178787:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.178788:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510750208:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.178802:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.178863:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510749440:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.178879:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.178904:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510750592:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.178916:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.178935:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.178937:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510751104:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.178953:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.178953:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.178974:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510750464:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.178975:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.178986:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.178991:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.179006:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510749824:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.179008:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.179017:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.179022:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.179036:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.179038:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982208 00000100:00100000:2.0:1785220582.179043:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982208 00000100:00100000:3.0:1785220582.179048:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.179062:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984000 00000100:00100000:3.0:1785220582.179065:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984000 00010000:00010000:2.0:1785220582.179076:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009b7c4b22/0xf62499ab9279316b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f753 expref: 124 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179078:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001701cb7a/0xf62499ab927918a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f51c expref: 156 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.179145:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982464 00000100:00100000:3.0:1785220582.179148:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982464 00010000:00010000:3.0:1785220582.179162:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000096c8b2ae/0xf62499ab92792e38 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f71b expref: 125 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.179181:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982720 00000100:00100000:3.0:1785220582.179184:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982720 00000100:00100000:1.0:1785220582.179194:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982976 00010000:00010000:3.0:1785220582.179196:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000498f31a9/0xf62499ab92792c32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f1 expref: 126 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.179197:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982976 00010000:00010000:1.0:1785220582.179211:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002b8877a1/0xf62499ab927914a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c8 expref: 157 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.179213:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519983232 00000100:00100000:3.0:1785220582.179216:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519983232 00010000:00010000:3.0:1785220582.179227:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003bb1175e/0xf62499ab927916c2 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f2 expref: 158 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.179230:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100587 00000100:00100000:1.0:1785220582.179238:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa63800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2406:x1871938519984000:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.179241:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519983488 00000100:00100000:3.0:1785220582.179243:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519983488 00010000:00010000:3.0:1785220582.179252:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001e70c6f2/0xf62499ab927913ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ba expref: 159 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179252:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.179258:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001701cb7a/0xf62499ab927918a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f51c expref: 159 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.179267:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519983744 00000100:00100000:3.0:1785220582.179269:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519983744 00010000:00010000:1.0:1785220582.179269:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001701cb7a/0xf62499ab927918a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f51c expref: 159 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179278:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000060d06c4f/0xf62499ab9279176a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f500 expref: 160 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179292:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001701cb7a/0xf62499ab927918a5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f51c expref: 160 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.179293:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100588 00000100:00100000:3.0:1785220582.179299:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e12450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+126:2404:x1871938519982464:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.179307:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.179311:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000096c8b2ae/0xf62499ab92792e38 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f71b expref: 126 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179315:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.179320:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000096c8b2ae/0xf62499ab92792e38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f71b expref: 126 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179320:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.179325:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179332:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.179335:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179336:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.179340:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179346:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179355:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.179356:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.179399:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e12450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+126:2404:x1871938519982464:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.179406:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100588 00000100:00100000:3.0:1785220582.179425:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100589 00000100:00100000:3.0:1785220582.179432:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+125:2404:x1871938519982720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.179432:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510750336:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.179440:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.179444:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000498f31a9/0xf62499ab92792c32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f1 expref: 125 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.179447:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179459:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000498f31a9/0xf62499ab92792c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f1 expref: 125 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.179469:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0000_UUID:14210:1871938510750720:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.179471:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000498f31a9/0xf62499ab92792c32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f1 expref: 125 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.179480:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179482:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.179486:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.179490:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179497:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.179499:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510750976:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.179509:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.179513:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.179537:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510751232:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.179549:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.179573:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100590 00000100:00100000:0.0:1785220582.179580:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+159:2404:x1871938519982976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.179588:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.179593:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002b8877a1/0xf62499ab927914a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c8 expref: 159 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.179604:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002b8877a1/0xf62499ab927914a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c8 expref: 159 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.179617:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b8877a1/0xf62499ab927914a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4c8 expref: 159 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.179628:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.179631:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.179635:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.179642:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.179661:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.179698:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa63800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+158:2406:x1871938519984000:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 461us (743us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.179705:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100587 00000100:00100000:1.0:1785220582.179720:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100591 00000100:00100000:1.0:1785220582.179726:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+157:2404:x1871938519983232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.179733:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.179738:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003bb1175e/0xf62499ab927916c2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f2 expref: 157 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179748:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003bb1175e/0xf62499ab927916c2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f2 expref: 157 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179771:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bb1175e/0xf62499ab927916c2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4f2 expref: 157 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179786:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.179789:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.179793:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179800:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179818:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.179838:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+156:2404:x1871938519983232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 112us (814us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.179844:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100591 00000100:00100000:1.0:1785220582.179857:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100592 00000100:00100000:1.0:1785220582.179864:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+155:2404:x1871938519983488:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.179871:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.179876:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001e70c6f2/0xf62499ab927913ce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ba expref: 155 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179886:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001e70c6f2/0xf62499ab927913ce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ba expref: 155 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179899:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001e70c6f2/0xf62499ab927913ce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4ba expref: 155 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179914:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.179917:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.179920:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179927:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.179943:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.179961:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+154:2404:x1871938519983488:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (923us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.179967:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+124:2404:x1871938519982720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 535us (974us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.179967:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100592 00000100:00100000:3.0:1785220582.179973:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100589 00000100:00100000:1.0:1785220582.179982:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100593 00000100:00100000:1.0:1785220582.180029:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+153:2404:x1871938519983744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.180038:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.180043:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000060d06c4f/0xf62499ab9279176a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f500 expref: 153 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180052:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000060d06c4f/0xf62499ab9279176a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f500 expref: 153 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180064:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000060d06c4f/0xf62499ab9279176a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f500 expref: 153 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180078:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.180081:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.180116:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180125:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180144:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.180164:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+152:2404:x1871938519983744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 176us (1114us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.180303:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+152:2404:x1871938519982976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 722us (1292us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.180349:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100594 00000100:00100000:3.0:1785220582.180357:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa60a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+123:2404:x1871938519982208:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.180365:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.180370:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009b7c4b22/0xf62499ab9279316b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f753 expref: 123 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.180377:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100593 00010000:00010000:3.0:1785220582.180385:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009b7c4b22/0xf62499ab9279316b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f753 expref: 123 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180398:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b7c4b22/0xf62499ab9279316b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f753 expref: 123 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180402:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180408:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.180410:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180411:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.180415:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180420:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180422:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.180429:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180440:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.180444:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.180458:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa60a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+122:2404:x1871938519982208:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (1524us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.180464:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100594 00010000:00010000:3.0:1785220582.180507:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180515:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180523:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180531:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.180545:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.180616:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1303:3652:x1871938510750848:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8627us (8675us total) trans 4295070152 rc 0/0 00000100:00100000:1.0:1785220582.180625:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284667 00010000:00010000:2.0:1785220582.180675:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.180684:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.180692:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.180704:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.180718:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.180845:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100590 00010000:00010000:0.0:1785220582.180872:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.180879:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.180887:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.180893:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.180904:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220582.181053:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1304:3652:x1871938510748288:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9711us (12161us total) trans 4295070153 rc 0/0 00000100:00100000:3.0:1785220582.181060:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284660 00000100:00100000:1.0:1785220582.181073:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510750080:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.181156:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181167:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181174:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181182:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181191:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1785220582.181191:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1303:3654:x1871938510747904:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9521us (10130us total) trans 4295070154 rc 0/0 00000100:00100000:2.0:1785220582.181198:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284664 00010000:00010000:2.0:1785220582.181217:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181226:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181233:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181241:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181248:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:3.0:1785220582.181278:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+110:3653:x1871938510747264:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11513us (12258us total) trans 4295070731 rc 0/0 00000100:00100000:3.0:1785220582.181285:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284658 00000100:00100000:2.0:1785220582.181319:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+110:3655:x1871938510748672:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11963us (12800us total) trans 4295070732 rc 0/0 00000100:00100000:2.0:1785220582.181325:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284655 00000100:00100000:3.0:1785220582.181340:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.181342:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:2.0:1785220582.181347:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181357:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.181361:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181369:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181384:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181391:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a56c3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.181396:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.181409:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1785220582.181412:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.181413:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982336 00000100:00100000:3.0:1785220582.181417:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982336 00010000:00010000:2.0:1785220582.181418:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.181421:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181428:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181436:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181437:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b97d6236/0xf62499ab9279300d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f737 expref: 122 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181445:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010cc6367/0xf62499ab927a5692 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.181455:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982592 00000100:00100000:3.0:1785220582.181457:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982592 00010000:00010000:2.0:1785220582.181457:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220582.181460:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181466:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.181470:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ed3b0cd1/0xf62499ab92791621 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4e4 expref: 151 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181470:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181478:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.181487:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100595 00010000:00010000:2.0:1785220582.181488:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.181493:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+122:2404:x1871938519982336:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.181496:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a56ae lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181502:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.181507:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b97d6236/0xf62499ab9279300d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f737 expref: 122 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181508:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.181512:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181518:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.181519:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b97d6236/0xf62499ab9279300d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f737 expref: 122 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181522:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181529:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181533:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b97d6236/0xf62499ab9279300d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f737 expref: 122 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181537:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181544:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.181544:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a56bc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181547:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.181552:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181556:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.181559:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181566:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181566:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.181570:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181577:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.181584:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.181586:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181592:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a5684 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.181604:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+121:2404:x1871938519982336:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 111us (264us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.181605:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.181608:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181614:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.181617:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181624:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181634:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181640:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a566f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.181653:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.181655:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.181656:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.181701:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.181718:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.181723:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.181732:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.181739:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.181748:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.181767:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.181780:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.181859:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+110:3655:x1871938510748160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13124us (13423us total) trans 4295070733 rc 0/0 00000100:00100000:1.0:1785220582.181866:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284654 00000100:00100000:0.0:1785220582.181950:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1302:3654:x1871938510749056:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10064us (10127us total) trans 4295070155 rc 0/0 00000100:00100000:0.0:1785220582.181957:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284666 00010000:00010000:0.0:1785220582.181992:0:14159:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096c8b2ae/0xf62499ab92792e38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f71b expref: 121 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.182003:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510749568:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.182016:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182025:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182036:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182044:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182051:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182066:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.182163:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+110:3652:x1871938510747648:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12669us (13583us total) trans 4295070734 rc 0/0 00000100:00100000:0.0:1785220582.182170:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284656 00010000:00010000:0.0:1785220582.182186:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.182190:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182198:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.182201:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182208:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182219:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182226:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a5668 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182239:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.182242:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182247:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.182250:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182256:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182265:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182271:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5676 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182284:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.182285:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.182287:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.182297:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182311:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182320:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182327:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182340:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.182415:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1302:3653:x1871938510748928:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10914us (11423us total) trans 4295070156 rc 0/0 00000100:00100000:0.0:1785220582.182422:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284662 00010000:00010000:0.0:1785220582.182439:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.182444:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182451:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.182454:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182461:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182473:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182480:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a56a0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.182493:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.182494:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.182496:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.182517:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510749952:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.182530:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.182989:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100595 00000100:00100000:2.0:1785220582.183011:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100596 00000100:00100000:2.0:1785220582.183040:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90703c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+151:2404:x1871938519982592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.183053:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.183060:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ed3b0cd1/0xf62499ab92791621 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4e4 expref: 151 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183076:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ed3b0cd1/0xf62499ab92791621 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4e4 expref: 151 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183118:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.183122:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.183128:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183137:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183151:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.183171:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90703c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+151:2404:x1871938519982592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 153us (1775us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.183179:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100596 00010000:00010000:2.0:1785220582.183216:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.183236:0:13836:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ed3b0cd1/0xf62499ab92791621 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4e4 expref: 150 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.183248:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510749696:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.183259:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183266:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183272:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183280:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183286:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183301:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.183396:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1302:3652:x1871938510748800:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13785us (14433us total) trans 4295070157 rc 0/0 00000100:00100000:2.0:1785220582.183403:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284657 00000004:00080000:2.0:1785220582.183439:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d99065f80/2, rc 0, transno 4295070152 00000100:00100000:2.0:1785220582.183450:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d99065f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510750848:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.183463:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72c700/2, rc 0, transno 4295070734 00000100:00100000:2.0:1785220582.183472:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510747648:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.183478:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.183510:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.183527:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519982848 00000100:00100000:3.0:1785220582.183531:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519982848 00000004:00080000:2.0:1785220582.183545:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72d180/2, rc 0, transno 4295070153 00000100:00100000:2.0:1785220582.183552:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510748288:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.183553:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a83276ce/0xf62499ab92792f26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f729 expref: 120 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.183560:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72e680/2, rc 0, transno 4295070157 00000100:00100000:2.0:1785220582.183563:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510748800:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.183573:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519983104 00000100:00100000:3.0:1785220582.183576:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519983104 00010000:00010000:3.0:1785220582.183588:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000095286827/0xf62499ab9279156b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4d6 expref: 150 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.183605:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510751872:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.183606:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100597 00000100:00100000:2.0:1785220582.183639:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.183667:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510751872 00000100:00100000:2.0:1785220582.183671:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510751872 00000100:00100000:3.0:1785220582.183692:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+120:2404:x1871938519982848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220582.183696:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284670 00000100:00100000:2.0:1785220582.183701:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1302:3652:x1871938510751872:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.183702:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.183707:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a83276ce/0xf62499ab92792f26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f729 expref: 120 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.183721:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29391 count 1 00010000:00010000:3.0:1785220582.183722:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a83276ce/0xf62499ab92792f26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f729 expref: 120 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183735:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a83276ce/0xf62499ab92792f26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f729 expref: 120 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183741:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183749:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.183763:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.183764:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000460fe600/0xf62499ab927919a8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f538 expref: 150 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183768:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183772:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72cf:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.183775:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.183777:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183779:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183796:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.183800:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000460fe600/0xf62499ab927919a8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f538 expref: 150 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.183811:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000460fe600/0xf62499ab927919a8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f538 expref: 150 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.183812:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+119:2404:x1871938519982848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 201us (335us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.183819:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100597 00000100:00100000:2.0:1785220582.183823:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510752128:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.183858:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.183877:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.183883:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510752000:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.183901:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100598 00000100:00100000:3.0:1785220582.183908:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+150:2404:x1871938519983104:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220582.183908:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.183917:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.183921:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000095286827/0xf62499ab9279156b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4d6 expref: 150 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183937:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000095286827/0xf62499ab9279156b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4d6 expref: 150 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183950:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095286827/0xf62499ab9279156b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f4d6 expref: 150 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.183957:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93d7ce00/2, rc 0, transno 4295070731 00010000:00010000:3.0:1785220582.183961:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.183963:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72c8:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220582.183964:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93d7ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510747264:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.183968:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183975:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.183995:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1785220582.183997:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72ea00/2, rc 0, transno 4295070156 00000100:00100000:2.0:1785220582.184004:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510748928:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.184009:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+149:2404:x1871938519983104:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (500us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.184016:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100598 00000100:00100000:2.0:1785220582.184034:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510751616:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.184059:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.184068:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.184116:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98b4b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510752256:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.184141:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.184164:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72c380/2, rc 0, transno 4295070154 00000100:00100000:2.0:1785220582.184172:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510747904:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.184203:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510752512:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.184225:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.184260:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72f100/2, rc 0, transno 4295070155 00000100:00100000:2.0:1785220582.184267:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510749056:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.184296:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.184303:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510752640:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.184309:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510752000 00000100:00100000:3.0:1785220582.184313:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510752000 00000100:00100000:2.0:1785220582.184325:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.184490:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510751616 00000100:00100000:3.0:1785220582.184494:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510751616 00000100:00100000:3.0:1785220582.184514:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510752256 00000100:00100000:3.0:1785220582.184517:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510752256 00000100:00100000:0.0:1785220582.184526:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519983360 00000100:00100000:0.0:1785220582.184530:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519983360 00000100:00100000:3.0:1785220582.184531:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510752512 00000100:00100000:3.0:1785220582.184532:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510752512 00000100:00100000:3.0:1785220582.184546:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510752640 00000100:00100000:3.0:1785220582.184548:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510752640 00010000:00010000:0.0:1785220582.184548:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000632d6a7b/0xf62499ab92792ccc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ff expref: 119 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.184562:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284671 00000100:00100000:3.0:1785220582.184568:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1305:3652:x1871938510752000:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.184568:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100599 00000100:00100000:0.0:1785220582.184573:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+119:2404:x1871938519983360:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.184580:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00080000:3.0:1785220582.184583:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29392 count 1 00010000:00010000:0.0:1785220582.184585:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000632d6a7b/0xf62499ab92792ccc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ff expref: 119 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184594:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000632d6a7b/0xf62499ab92792ccc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ff expref: 119 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.184596:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184613:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000632d6a7b/0xf62499ab92792ccc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ff expref: 119 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184623:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.184627:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.184631:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184638:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184656:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.184682:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007fc135f1/0xf62499ab92791a34 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f546 expref: 148 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.184692:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d0:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.184695:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.184699:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.184699:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284672 00000100:00100000:1.0:1785220582.184708:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+111:3654:x1871938510751616:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.184718:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007fc135f1/0xf62499ab92791a34 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f546 expref: 148 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.184721:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29432 count 1 00010000:00010000:3.0:1785220582.184727:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007fc135f1/0xf62499ab92791a34 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f546 expref: 148 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.184733:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.184737:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c680a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510752768:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.184744:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b53c6b14/0xf62499ab92793244 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f76f expref: 118 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.184761:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f8:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.184765:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.184793:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.184820:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b53c6b14/0xf62499ab92793244 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f76f expref: 118 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.184830:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b53c6b14/0xf62499ab92793244 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f76f expref: 118 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.184840:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510752896:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.184879:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+118:2404:x1871938519983360:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 305us (810us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.184887:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100599 00010000:00010000:0.0:1785220582.184908:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1785220582.184909:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a5a9d80 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.184914:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184923:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:3.0:1785220582.184925:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284673 00010000:00010000:0.0:1785220582.184927:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.184932:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+111:3654:x1871938510752256:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.184936:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220582.184944:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29433 count 1 00010000:00010000:0.0:1785220582.184953:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.184955:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184962:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a567d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.184966:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007d0df699/0xf62499ab92793298 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f77d expref: 117 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.184972:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72f9:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.184975:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.184978:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.184979:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.184980:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.184982:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.184997:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284674 00010000:00010000:3.0:1785220582.184998:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007d0df699/0xf62499ab92793298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f77d expref: 117 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185004:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1305:3652:x1871938510752512:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.185007:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007d0df699/0xf62499ab92793298 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f77d expref: 117 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.185014:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29394 count 1 00000100:00100000:3.0:1785220582.185016:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510753024:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.185025:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185033:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.185036:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000037b8f9b6/0xf62499ab92791bb5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f562 expref: 148 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185043:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d2:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.185046:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.185049:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185065:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000037b8f9b6/0xf62499ab92791bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f562 expref: 148 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185079:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000037b8f9b6/0xf62499ab92791bb5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f562 expref: 148 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185120:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510753152:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.185141:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.185157:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185165:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185175:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185182:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185196:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.185318:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+112:3653:x1871938510747776:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14293us (16260us total) trans 4295070735 rc 0/0 00000100:00100000:0.0:1785220582.185324:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284659 00000100:00100000:0.0:1785220582.185339:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284675 00000100:00100000:0.0:1785220582.185346:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1305:3652:x1871938510752640:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.185355:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29395 count 1 00010000:00010000:0.0:1785220582.185366:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185377:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007275bfa9/0xf62499ab92791c3a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f570 expref: 148 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185383:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.185384:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d3:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.185387:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.185391:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185406:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519983616 00010000:00010000:0.0:1785220582.185409:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007275bfa9/0xf62499ab92791c3a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f570 expref: 148 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185410:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519983616 00010000:00010000:0.0:1785220582.185417:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007275bfa9/0xf62499ab92791c3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f570 expref: 148 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.185426:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007ef91bf3/0xf62499ab9279308b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f745 expref: 118 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185427:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510753280:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.185445:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.185453:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100600 00010000:00010000:0.0:1785220582.185454:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.185459:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185460:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+118:2404:x1871938519983616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.185467:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.185467:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.185470:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.185472:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007ef91bf3/0xf62499ab9279308b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f745 expref: 118 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.185482:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007ef91bf3/0xf62499ab9279308b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f745 expref: 118 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185484:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185495:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.185496:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ef91bf3/0xf62499ab9279308b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f745 expref: 118 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185503:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d9f8f45c/0xf62499ab927a568b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.185507:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.185510:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.185514:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185516:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.185518:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.185520:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.185522:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185545:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510751744:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.185547:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.185556:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185560:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+117:2404:x1871938519983616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (474us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.185564:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185567:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100600 00010000:00010000:0.0:1785220582.185573:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185580:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185598:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.185675:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+112:3654:x1871938510748544:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13959us (14176us total) trans 4295070736 rc 0/0 00000100:00100000:0.0:1785220582.185682:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284665 00000100:00100000:3.0:1785220582.185702:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.185717:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.185719:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519983872 00000100:00100000:3.0:1785220582.185723:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519983872 00000100:00100000:0.0:1785220582.185736:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510752384:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.185739:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e6099ad4/0xf62499ab927917f6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f50e expref: 149 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185769:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.185770:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100601 00000100:00100000:3.0:1785220582.185777:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+149:2404:x1871938519983872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.185784:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.185789:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e6099ad4/0xf62499ab927917f6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f50e expref: 149 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.185795:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72c000/2, rc 0, transno 4295070732 00010000:00010000:3.0:1785220582.185798:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e6099ad4/0xf62499ab927917f6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f50e expref: 149 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185806:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510748672:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.185810:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e6099ad4/0xf62499ab927917f6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f50e expref: 149 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.185819:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cc:0x0].0x0 since=0s v=(1 0) 00000004:00080000:0.0:1785220582.185847:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72df80/2, rc 0, transno 4295070733 00000100:00100000:0.0:1785220582.185854:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510748160:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.185881:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510753536:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.185897:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.185903:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185903:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.185911:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.185921:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.185925:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510753408:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.185940:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.185945:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185954:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+148:2404:x1871938519983872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 178us (252us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.185955:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.185961:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100601 00010000:00010000:0.0:1785220582.185963:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185976:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.185988:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.186066:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.186070:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1306:3654:x1871938510749312:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13483us (13695us total) trans 4295070158 rc 0/0 00000100:00100000:0.0:1785220582.186077:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284668 00000100:00100000:0.0:1785220582.186108:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510751744 00000100:00100000:0.0:1785220582.186112:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510751744 00000100:00100000:0.0:1785220582.186134:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510752384 00000100:00100000:0.0:1785220582.186137:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510752384 00000100:00100000:0.0:1785220582.186152:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510753536 00000100:00100000:0.0:1785220582.186155:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510753536 00000100:00100000:0.0:1785220582.186172:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284676 00000100:00100000:0.0:1785220582.186177:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1307:3655:x1871938510751744:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.186190:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29390 count 1 00010000:00010000:0.0:1785220582.186202:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186214:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003043439c/0xf62499ab927918f2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f52a expref: 147 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186221:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ce:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.186224:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.186228:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186252:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003043439c/0xf62499ab927918f2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f52a expref: 147 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186262:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003043439c/0xf62499ab927918f2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f52a expref: 147 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.186273:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510753664:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.186291:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.186402:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.186407:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186415:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.186418:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186428:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186440:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186446:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fa329aa/0xf62499ab927a56b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186465:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.186468:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186474:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.186477:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186484:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186495:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186502:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a56ca lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186512:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.186514:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.186515:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.186530:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284677 00000100:00100000:0.0:1785220582.186537:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1307:3655:x1871938510752384:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.186551:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29393 count 1 00010000:00010000:0.0:1785220582.186562:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186580:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003c6e78fd/0xf62499ab92791b06 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f554 expref: 147 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186585:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d1:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.186589:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.186592:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186612:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003c6e78fd/0xf62499ab92791b06 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f554 expref: 147 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186622:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003c6e78fd/0xf62499ab92791b06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f554 expref: 147 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.186631:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510753792:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.186654:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.186671:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284678 00000100:00100000:0.0:1785220582.186679:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+112:3655:x1871938510753536:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.186691:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29435 count 1 00010000:00010000:0.0:1785220582.186702:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186712:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e8dc2930/0xf62499ab927932b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f799 expref: 116 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186723:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72fb:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.186725:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.186728:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186744:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e8dc2930/0xf62499ab927932b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f799 expref: 116 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.186763:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e8dc2930/0xf62499ab927932b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f799 expref: 116 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.186773:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510753920:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.186795:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.186822:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.186975:0:14144:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510753408 00000100:00100000:2.0:1785220582.186980:0:14144:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510753408 00000100:00100000:2.0:1785220582.187053:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284679 00000100:00100000:2.0:1785220582.187061:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+113:3655:x1871938510753408:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.187073:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29434 count 1 00000100:00100000:1.0:1785220582.187283:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba780 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.187327:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.187343:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c76b052a/0xf62499ab927932a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f78b expref: 116 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.187351:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72fa:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.187354:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.187358:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.187380:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c76b052a/0xf62499ab927932a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f78b expref: 116 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.187389:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c76b052a/0xf62499ab927932a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f78b expref: 116 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.187398:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510754048:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.187426:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6540 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.187527:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510751488:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.187546:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.187566:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984128 00000100:00100000:1.0:1785220582.187570:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984128 00010000:00010000:1.0:1785220582.187588:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b1e786f5/0xf62499ab927931b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f761 expref: 117 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.188018:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100602 00000100:00100000:1.0:1785220582.188029:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+117:2406:x1871938519984128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.188038:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.188044:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b1e786f5/0xf62499ab927931b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f761 expref: 117 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.188055:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b1e786f5/0xf62499ab927931b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f761 expref: 117 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.188071:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1e786f5/0xf62499ab927931b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f761 expref: 117 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.188083:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.188150:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.188155:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.188164:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.188187:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.188214:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+116:2406:x1871938519984128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 187us (2148us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.188222:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100602 00010000:00010000:2.0:1785220582.188292:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.188304:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188314:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188323:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188331:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188348:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.188502:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+114:3654:x1871938510747392:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16952us (17478us total) trans 4295070737 rc 0/0 00000100:00100000:2.0:1785220582.188510:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284663 00010000:00010000:2.0:1785220582.188531:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.188536:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188544:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.188547:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188557:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188568:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188574:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a56a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188606:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.188609:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.188611:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.188661:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72e300/2, rc 0, transno 4295070735 00000100:00100000:2.0:1785220582.188674:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510747776:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.188727:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510754176:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.188769:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.188795:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510754176 00000100:00100000:2.0:1785220582.188799:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510754176 00000100:00100000:2.0:1785220582.188826:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284680 00000100:00100000:2.0:1785220582.188832:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+114:3652:x1871938510754176:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.188851:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29436 count 1 00010000:00010000:2.0:1785220582.188869:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188883:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bf849d72/0xf62499ab927932c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a7 expref: 115 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188891:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72fc:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.188894:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.188898:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188919:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bf849d72/0xf62499ab927932c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a7 expref: 115 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.188930:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bf849d72/0xf62499ab927932c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a7 expref: 115 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.188948:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510754304:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.188984:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6c00 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.189042:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915af800/2, rc 0, transno 4295070158 00000100:00100000:2.0:1785220582.189053:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510749312:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.189108:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.189116:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510754432:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.189140:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.189173:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72ed80/2, rc 0, transno 4295070737 00000100:00100000:2.0:1785220582.189181:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510747392:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.189215:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72f480/2, rc 0, transno 4295070736 00000100:00100000:2.0:1785220582.189222:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510748544:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.189247:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510754560:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.189269:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.189286:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510754432 00000100:00100000:2.0:1785220582.189290:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510754432 00000100:00100000:2.0:1785220582.189322:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510754560 00000100:00100000:2.0:1785220582.189325:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510754560 00000100:00100000:2.0:1785220582.189345:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284681 00000100:00100000:2.0:1785220582.189351:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b0e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1308:3652:x1871938510754432:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.189364:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29396 count 1 00010000:00010000:2.0:1785220582.189380:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189392:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001a6739ef/0xf62499ab92791cb8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f57e expref: 147 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189399:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d4:0x0].0x0 since=19s v=(0 0) 00010000:00010000:2.0:1785220582.189402:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.189406:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189425:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001a6739ef/0xf62499ab92791cb8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f57e expref: 147 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189435:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001a6739ef/0xf62499ab92791cb8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f57e expref: 147 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.189446:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510754816:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.189481:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d63c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.189498:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284682 00000100:00100000:2.0:1785220582.189505:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+115:3654:x1871938510754560:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.189518:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29437 count 1 00010000:00010000:2.0:1785220582.189529:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189540:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b09056f0/0xf62499ab927932d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7b5 expref: 115 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189547:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72fd:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.189550:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.189554:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189572:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b09056f0/0xf62499ab927932d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7b5 expref: 115 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.189580:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b09056f0/0xf62499ab927932d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7b5 expref: 115 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.189590:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510754944:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.189611:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.189625:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.189630:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89effb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510754688:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.189655:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.190482:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190495:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190505:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190513:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190546:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.190713:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1309:3653:x1871938510748416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19365us (20795us total) trans 4295070159 rc 0/0 00000100:00100000:2.0:1785220582.190721:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284661 00000100:00100000:2.0:1785220582.190744:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510754688 00000100:00100000:2.0:1785220582.190760:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510754688 00000100:00100000:2.0:1785220582.190790:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284683 00000100:00100000:2.0:1785220582.190796:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97adf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+116:3654:x1871938510754688:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.190813:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29438 count 1 00010000:00010000:2.0:1785220582.190828:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190843:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e7656ce5/0xf62499ab927932de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7c3 expref: 115 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190851:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72fe:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.190855:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.190859:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190883:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e7656ce5/0xf62499ab927932de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7c3 expref: 115 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.190894:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e7656ce5/0xf62499ab927932de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7c3 expref: 115 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.190911:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510755072:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.190950:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.190975:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510752128:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.190976:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c680a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510752768:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.190994:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.190995:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.191037:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.191062:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984256 00000100:00100000:0.0:1785220582.191067:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984256 00010000:00010000:0.0:1785220582.191076:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.191095:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.191116:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.191117:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.191121:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191131:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.191142:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984384 00000100:00100000:3.0:1785220582.191145:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984384 00010000:00010000:0.0:1785220582.191151:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191158:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d42bdce/0xf62499ab927a5699 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191164:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007fc135f1/0xf62499ab92791a34 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f546 expref: 149 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191180:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.191182:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.191184:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.191186:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100603 00000100:00100000:3.0:1785220582.191192:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+149:2406:x1871938519984384:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.191202:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.191207:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007fc135f1/0xf62499ab92791a34 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f546 expref: 149 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191218:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007fc135f1/0xf62499ab92791a34 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f546 expref: 149 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191219:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000460fe600/0xf62499ab927919a8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f538 expref: 149 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.191239:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100604 00010000:00010000:3.0:1785220582.191240:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007fc135f1/0xf62499ab92791a34 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f546 expref: 149 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.191245:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+149:2407:x1871938519984256:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.191251:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.191254:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.191255:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.191258:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000460fe600/0xf62499ab927919a8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f538 expref: 148 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191260:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191267:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000460fe600/0xf62499ab927919a8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f538 expref: 148 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191269:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191279:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000460fe600/0xf62499ab927919a8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f538 expref: 148 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191288:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.191293:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.191295:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.191298:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191299:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191306:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191308:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191315:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191322:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191323:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.191334:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.191345:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+147:2407:x1871938519984256:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.191351:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100604 00000100:00100000:3.0:1785220582.191427:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1309:3652:x1871938510752000:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6858us (7518us total) trans 4295070160 rc 0/0 00000100:00100000:3.0:1785220582.191434:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284671 00000100:00100000:3.0:1785220582.191465:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+146:2406:x1871938519984384:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 273us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.191471:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100603 00000100:00100000:0.0:1785220582.191574:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510753152:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.191584:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.191588:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.191601:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.191611:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0001_UUID:14222:1871938510753280:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.191622:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.191643:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510753664:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.191655:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.191676:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984640 00000100:00100000:0.0:1785220582.191679:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984640 00010000:00010000:0.0:1785220582.191693:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000037b8f9b6/0xf62499ab92791bb5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f562 expref: 146 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.191712:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984896 00000100:00100000:0.0:1785220582.191715:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984896 00010000:00010000:0.0:1785220582.191726:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003043439c/0xf62499ab927918f2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f52a expref: 147 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.191745:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100605 00000100:00100000:3.0:1785220582.191762:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100606 00000100:00100000:0.0:1785220582.191762:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+147:2406:x1871938519984640:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.191770:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa63800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+147:2406:x1871938519984896:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.191770:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.191774:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000037b8f9b6/0xf62499ab92791bb5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f562 expref: 147 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191776:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.191781:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003043439c/0xf62499ab927918f2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f52a expref: 147 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191785:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000037b8f9b6/0xf62499ab92791bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f562 expref: 147 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191790:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003043439c/0xf62499ab927918f2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f52a expref: 147 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191798:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000037b8f9b6/0xf62499ab92791bb5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f562 expref: 147 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191801:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003043439c/0xf62499ab927918f2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f52a expref: 147 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191809:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.191810:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.191812:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.191813:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.191817:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191817:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.191825:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191826:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191842:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.191842:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.191859:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa63800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+145:2406:x1871938519984896:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.191866:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100606 00010000:00010000:3.0:1785220582.191883:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191891:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191899:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191906:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.191919:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.191931:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.191940:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.191948:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.191955:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.191967:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220582.192034:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.192039:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192047:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.192051:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192058:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192072:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192078:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a56df lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.192272:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+144:2406:x1871938519984640:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 519us (685us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.192280:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100605 00010000:00010000:0.0:1785220582.192300:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.192308:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.192314:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.192321:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.192330:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.192410:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+1310:3652:x1871938510752512:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7406us (8183us total) trans 4295070162 rc 0/0 00000100:00100000:0.0:1785220582.192416:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284674 00000100:00100000:0.0:1785220582.192450:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510753024:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.192466:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192519:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.192524:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192530:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.192533:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192539:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192550:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192556:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5702 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192580:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.192584:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192590:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.192594:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192601:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192617:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192625:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a56d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192640:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.192644:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192650:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.192653:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192660:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192670:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192677:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a56f4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192692:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.192694:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.192696:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.192707:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192714:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192722:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192729:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192745:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.192842:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b108700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1310:3652:x1871938510751872:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9140us (9202us total) trans 4295070163 rc 0/0 00000100:00100000:2.0:1785220582.192849:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284670 00010000:00010000:2.0:1785220582.192868:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.192872:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192878:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.192881:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192888:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192903:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192909:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a56d8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.192923:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.192925:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.192926:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.192934:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.192987:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efe300/2, rc 0, transno 4295070162 00000100:00100000:2.0:1785220582.192999:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510752512:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.193010:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10b480/2, rc 0, transno 4295070160 00000100:00100000:2.0:1785220582.193014:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510752000:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.193021:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b108000/2, rc 0, transno 4295070163 00000100:00100000:2.0:1785220582.193025:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510751872:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.193274:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510755200:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.193303:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1309:3655:x1871938510751744:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7124us (7583us total) trans 4295070161 rc 0/0 00000100:00100000:2.0:1785220582.193306:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.193311:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284676 00000100:00100000:3.0:1785220582.193326:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510755200 00000100:00100000:3.0:1785220582.193330:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510755200 00000100:00100000:2.0:1785220582.193331:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.193336:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510755328:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.193355:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.193359:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284684 00000100:00100000:3.0:1785220582.193366:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97ade680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1309:3652:x1871938510755200:12345-0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.193383:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72f800/2, rc 0, transno 4295070159 00002000:00080000:3.0:1785220582.193388:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29397 count 1 00000100:00100000:2.0:1785220582.193391:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510748416:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.193406:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.193421:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000558ff3eb/0xf62499ab92791d3d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f58c expref: 143 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.193424:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97add500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510755456:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.193429:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d5:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.193433:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.193437:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.193445:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.193461:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000558ff3eb/0xf62499ab92791d3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f58c expref: 143 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.193461:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.193465:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97addf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510755584:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.193474:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000558ff3eb/0xf62499ab92791d3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f58c expref: 143 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.193482:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.193485:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510755712:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.193524:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920c00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.193535:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.193565:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.193574:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510755328 00000100:00100000:0.0:1785220582.193577:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510755328 00000100:00100000:0.0:1785220582.193598:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510755456 00000100:00100000:0.0:1785220582.193600:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510755456 00000100:00100000:0.0:1785220582.193614:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510755584 00000100:00100000:0.0:1785220582.193617:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510755584 00000100:00100000:0.0:1785220582.193633:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284685 00000100:00100000:3.0:1785220582.193634:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.193639:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97ade300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1312:3652:x1871938510755328:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.193650:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29398 count 1 00010000:00010000:0.0:1785220582.193663:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193675:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009f0200c4/0xf62499ab92791dc2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f59a expref: 143 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193682:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d6:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.193685:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.193689:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193707:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009f0200c4/0xf62499ab92791dc2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f59a expref: 143 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193716:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009f0200c4/0xf62499ab92791dc2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f59a expref: 143 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.193726:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510755840:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.193772:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.193789:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284686 00000100:00100000:0.0:1785220582.193797:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97adc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1312:3654:x1871938510755456:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.193809:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29399 count 1 00010000:00010000:0.0:1785220582.193822:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193833:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003f264abb/0xf62499ab92791e5c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a8 expref: 143 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193840:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d7:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.193843:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.193847:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193865:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003f264abb/0xf62499ab92791e5c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a8 expref: 143 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193875:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003f264abb/0xf62499ab92791e5c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a8 expref: 143 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.193885:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510755968:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.193909:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9e40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.193927:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284687 00000100:00100000:0.0:1785220582.193934:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97add880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1312:3654:x1871938510755584:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.193946:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29400 count 1 00010000:00010000:0.0:1785220582.193958:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193969:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006fe71057/0xf62499ab92791f04 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5b6 expref: 143 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.193975:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d8:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.193978:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.193981:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194000:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006fe71057/0xf62499ab92791f04 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5b6 expref: 143 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194009:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006fe71057/0xf62499ab92791f04 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5b6 expref: 143 pid: 14187 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.194019:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510756096:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.194041:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.194055:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984512 00000100:00100000:0.0:1785220582.194058:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984512 00010000:00010000:0.0:1785220582.194078:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007d0df699/0xf62499ab92793298 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f77d expref: 116 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.194203:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519984768 00000100:00100000:2.0:1785220582.194207:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519984768 00010000:00010000:2.0:1785220582.194225:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007275bfa9/0xf62499ab92791c3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f570 expref: 144 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.194250:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100607 00000100:00100000:2.0:1785220582.194257:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+144:2406:x1871938519984768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.194266:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.194271:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007275bfa9/0xf62499ab92791c3a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f570 expref: 144 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.194282:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007275bfa9/0xf62499ab92791c3a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f570 expref: 144 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.194298:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007275bfa9/0xf62499ab92791c3a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f570 expref: 144 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.194310:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.194314:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.194318:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.194332:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.194350:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.194383:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+143:2406:x1871938519984768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 126us (748us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.194390:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100607 00000100:00100000:0.0:1785220582.194446:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100608 00000100:00100000:0.0:1785220582.194454:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+116:2406:x1871938519984512:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.194462:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.194467:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007d0df699/0xf62499ab92793298 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f77d expref: 116 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194478:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007d0df699/0xf62499ab92793298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f77d expref: 116 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194493:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d0df699/0xf62499ab92793298 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f77d expref: 116 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194505:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.194508:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.194512:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194519:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194544:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.194575:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+115:2406:x1871938519984512:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 122us (1010us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.194582:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100608 00010000:00010000:0.0:1785220582.194605:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194614:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194625:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194632:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.194643:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.194783:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+1313:3652:x1871938510752640:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9438us (10456us total) trans 4295070164 rc 0/0 00000100:00100000:0.0:1785220582.194790:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284675 00010000:00010000:3.0:1785220582.194796:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.194805:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.194818:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.194835:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.194841:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10b800/2, rc 0, transno 4295070161 00010000:00010000:3.0:1785220582.194849:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.194850:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510751744:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.194886:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.194895:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510756224:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.194922:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.194935:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+118:3654:x1871938510752256:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10004us (10793us total) trans 4295070739 rc 0/0 00000100:00100000:3.0:1785220582.194942:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284673 00000100:00100000:3.0:1785220582.194957:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510756224 00000100:00100000:3.0:1785220582.194961:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510756224 00000100:00100000:3.0:1785220582.194990:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284688 00000100:00100000:3.0:1785220582.194996:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1313:3653:x1871938510756224:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.195011:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29401 count 1 00010000:00010000:3.0:1785220582.195025:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.195036:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dcd58a22/0xf62499ab92791f9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5c4 expref: 142 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.195043:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72d9:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.195046:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.195050:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.195074:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dcd58a22/0xf62499ab92791f9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5c4 expref: 142 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.195103:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dcd58a22/0xf62499ab92791f9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5c4 expref: 142 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.195114:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510756352:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.195246:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.195252:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195260:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.195264:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195273:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195286:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195293:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000460532ec/0xf62499ab927a56fb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195311:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.195314:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195321:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.195324:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195331:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195340:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195347:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a56ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.195367:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.195368:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.195370:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.195480:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.197908:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8b10ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+117:3652:x1871938510751360:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23916us (24685us total) trans 4295070738 rc 0/0 00000100:00100000:1.0:1785220582.197922:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284669 00000100:00100000:3.0:1785220582.197939:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.197958:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.198230:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510753920:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.198256:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.198329:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efea00/2, rc 0, transno 4295070164 00000100:00100000:2.0:1785220582.198377:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510752640:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.198394:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb0380/2, rc 0, transno 4295070738 00000100:00100000:2.0:1785220582.198414:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510751360:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.198467:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98b4b100/2, rc 0, transno 4295070739 00000100:00100000:2.0:1785220582.198472:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98b4b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510752256:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.198591:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.198604:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510756480:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.198629:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0000_UUID:14257:1871938510752896:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.198643:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.198779:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.198849:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510756736:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.198874:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.198938:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.198944:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510756608:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.198966:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.199008:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510756480 00000100:00100000:0.0:1785220582.199015:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510756480 00000100:00100000:0.0:1785220582.199052:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510756736 00000100:00100000:0.0:1785220582.199056:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510756736 00000100:00100000:0.0:1785220582.199083:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284689 00000100:00100000:0.0:1785220582.199216:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+117:3652:x1871938510756480:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.199242:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29439 count 1 00010000:00010000:0.0:1785220582.199264:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199280:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000033044c95/0xf62499ab927932ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d1 expref: 114 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199288:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x72ff:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.199293:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.199298:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199321:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000033044c95/0xf62499ab927932ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d1 expref: 114 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199333:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000033044c95/0xf62499ab927932ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d1 expref: 114 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.199352:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510756864:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.199398:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ebd9ee600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.199416:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284690 00000100:00100000:0.0:1785220582.199424:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1314:3655:x1871938510756736:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.199437:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29402 count 1 00010000:00010000:0.0:1785220582.199451:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199463:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000a7c65e2/0xf62499ab9279201c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d2 expref: 142 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199468:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72da:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.199471:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.199475:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199493:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000a7c65e2/0xf62499ab9279201c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d2 expref: 142 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.199502:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000a7c65e2/0xf62499ab9279201c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d2 expref: 142 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.199513:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510756992:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.199538:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960ec1464780 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.199982:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510756608 00000100:00100000:0.0:1785220582.199987:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510756608 00000100:00100000:0.0:1785220582.200017:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284691 00000100:00100000:0.0:1785220582.200024:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+118:3655:x1871938510756608:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.200037:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29440 count 1 00010000:00010000:0.0:1785220582.200050:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.200062:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ff19f5e9/0xf62499ab927932fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7df expref: 114 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.200069:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7300:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.200072:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7300:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.200075:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.200231:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ff19f5e9/0xf62499ab927932fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7df expref: 114 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.200243:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ff19f5e9/0xf62499ab927932fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7df expref: 114 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.200255:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510757120:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.200290:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.200506:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510754048:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.200525:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.200675:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.200692:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510753792:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.200707:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.200708:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.200735:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.200763:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985152 00000100:00100000:3.0:1785220582.200768:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.200768:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985152 00010000:00010000:1.0:1785220582.200790:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b53c6b14/0xf62499ab92793244 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f76f expref: 115 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.200811:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985408 00000100:00100000:1.0:1785220582.200814:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985408 00000100:00100000:3.0:1785220582.200826:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.200837:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003c6e78fd/0xf62499ab92791b06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f554 expref: 143 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.200842:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.200853:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985024 00000100:00100000:3.0:1785220582.200856:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.200856:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985024 00010000:00010000:1.0:1785220582.200867:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e8dc2930/0xf62499ab927932b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f799 expref: 116 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.200873:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.200932:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510754304:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.200946:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.200987:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510754816:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.200998:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.201019:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510754944:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.201030:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.201197:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985280 00000100:00100000:1.0:1785220582.201201:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985280 00010000:00010000:1.0:1785220582.201216:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c76b052a/0xf62499ab927932a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f78b expref: 117 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.201240:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985536 00000100:00100000:1.0:1785220582.201244:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985536 00010000:00010000:1.0:1785220582.201256:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bf849d72/0xf62499ab927932c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a7 expref: 118 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.201273:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985792 00000100:00100000:1.0:1785220582.201276:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985792 00010000:00010000:1.0:1785220582.201288:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b09056f0/0xf62499ab927932d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7b5 expref: 119 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.201302:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985664 00000100:00100000:1.0:1785220582.201304:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985664 00010000:00010000:1.0:1785220582.201314:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001a6739ef/0xf62499ab92791cb8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f57e expref: 144 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.201329:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519985920 00000100:00100000:1.0:1785220582.201332:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519985920 00010000:00010000:1.0:1785220582.201341:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e7656ce5/0xf62499ab927932de lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7c3 expref: 120 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.201354:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100609 00000100:00100000:1.0:1785220582.201361:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+120:2406:x1871938519985152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.201374:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.201379:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b53c6b14/0xf62499ab92793244 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f76f expref: 120 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201394:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b53c6b14/0xf62499ab92793244 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f76f expref: 120 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201416:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b53c6b14/0xf62499ab92793244 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f76f expref: 120 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201430:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.201434:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.201440:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201449:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201473:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.201508:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+119:2406:x1871938519985152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 147us (834us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.201516:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100609 00000100:00100000:1.0:1785220582.201533:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100610 00000100:00100000:1.0:1785220582.201540:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e12c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+144:2406:x1871938519985408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.201548:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.201553:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003c6e78fd/0xf62499ab92791b06 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f554 expref: 144 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201567:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003c6e78fd/0xf62499ab92791b06 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f554 expref: 144 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201582:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c6e78fd/0xf62499ab92791b06 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f554 expref: 144 pid: 14205 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201593:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.201596:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.201599:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201606:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201624:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.201642:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e12c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+143:2406:x1871938519985408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (934us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.201649:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100610 00000100:00100000:1.0:1785220582.201686:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100611 00000100:00100000:1.0:1785220582.201693:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+118:2406:x1871938519985024:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.201700:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.201705:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e8dc2930/0xf62499ab927932b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f799 expref: 118 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201718:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e8dc2930/0xf62499ab927932b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f799 expref: 118 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201729:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8dc2930/0xf62499ab927932b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f799 expref: 118 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.201738:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.201740:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.201744:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202053:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202078:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.202130:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+117:2406:x1871938519985024:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 437us (1394us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.202137:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100611 00000100:00100000:1.0:1785220582.202152:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100612 00000100:00100000:1.0:1785220582.202159:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+116:2406:x1871938519985280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.202168:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.202173:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c76b052a/0xf62499ab927932a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f78b expref: 116 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202189:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c76b052a/0xf62499ab927932a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f78b expref: 116 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202201:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c76b052a/0xf62499ab927932a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f78b expref: 116 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202211:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202214:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202217:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202224:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202242:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.202261:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+115:2406:x1871938519985280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (1491us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.202268:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100612 00000100:00100000:1.0:1785220582.202282:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100613 00000100:00100000:1.0:1785220582.202289:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+114:2407:x1871938519985536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.202297:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.202302:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bf849d72/0xf62499ab927932c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a7 expref: 114 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202317:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bf849d72/0xf62499ab927932c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a7 expref: 114 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202330:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf849d72/0xf62499ab927932c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a7 expref: 114 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202340:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202343:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202347:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202353:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202370:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.202389:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+113:2407:x1871938519985536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (1563us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.202396:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100613 00000100:00100000:1.0:1785220582.202409:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100614 00000100:00100000:1.0:1785220582.202416:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+112:2407:x1871938519985792:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.202422:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.202427:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b09056f0/0xf62499ab927932d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7b5 expref: 112 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202442:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b09056f0/0xf62499ab927932d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7b5 expref: 112 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202454:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b09056f0/0xf62499ab927932d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7b5 expref: 112 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202463:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202466:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202470:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202476:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202494:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.202511:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+111:2407:x1871938519985792:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1668us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.202518:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100614 00000100:00100000:1.0:1785220582.202531:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100615 00000100:00100000:1.0:1785220582.202537:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+142:2407:x1871938519985664:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.202544:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.202548:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001a6739ef/0xf62499ab92791cb8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f57e expref: 142 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202562:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001a6739ef/0xf62499ab92791cb8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f57e expref: 142 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202574:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001a6739ef/0xf62499ab92791cb8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f57e expref: 142 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202584:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202587:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202591:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202598:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202616:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.202634:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+141:2407:x1871938519985664:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (1776us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.202640:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100615 00000100:00100000:1.0:1785220582.202653:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100616 00000100:00100000:1.0:1785220582.202659:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+110:2407:x1871938519985920:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.202667:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.202672:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e7656ce5/0xf62499ab927932de lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7c3 expref: 110 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202686:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e7656ce5/0xf62499ab927932de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7c3 expref: 110 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202698:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202701:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.202705:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202713:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202726:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.202745:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+110:2407:x1871938519985920:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (1871us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.202765:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100616 00010000:00010000:1.0:1785220582.202793:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202802:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202812:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202825:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.202844:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.203011:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+119:3655:x1871938510753536:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16331us (17104us total) trans 4295070740 rc 0/0 00000100:00100000:1.0:1785220582.203019:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284678 00010000:00010000:1.0:1785220582.203042:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203051:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203058:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203065:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203079:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.203185:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+119:3652:x1871938510754176:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14352us (14415us total) trans 4295070741 rc 0/0 00000100:00100000:1.0:1785220582.203192:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284680 00010000:00010000:1.0:1785220582.203213:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203221:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203233:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203239:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203254:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220582.203326:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+119:3654:x1871938510754560:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13821us (14055us total) trans 4295070742 rc 0/0 00000100:00100000:1.0:1785220582.203332:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284682 00010000:00010000:1.0:1785220582.203354:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203362:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203370:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203378:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203391:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:1.0:1785220582.203476:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b0e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1315:3652:x1871938510754432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14125us (14334us total) trans 4295070165 rc 0/0 00000100:00100000:1.0:1785220582.203486:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284681 00010000:00010000:1.0:1785220582.203506:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203515:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203524:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203532:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.203541:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:1.0:1785220582.203608:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f724000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+119:3654:x1871938510751616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18902us (19548us total) trans 4295070743 rc 0/0 00000100:00100000:1.0:1785220582.203614:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284672 00000004:00080000:1.0:1785220582.203674:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efca80/2, rc 0, transno 4295070165 00000100:00100000:1.0:1785220582.203686:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510754432:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.203699:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efce00/2, rc 0, transno 4295070741 00000100:00100000:1.0:1785220582.203707:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510754176:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.203732:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:3.0:1785220582.203739:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203747:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ada00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.203762:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203777:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.203779:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510757248:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.203791:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203798:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0447b82/0xf62499ab927a5710 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.203812:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.203816:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:3.0:1785220582.203820:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203826:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.203829:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203835:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203845:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203852:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a571e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.203852:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbca80/2, rc 0, transno 4295070740 00000100:00100000:1.0:1785220582.203859:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510753536:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.203869:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220582.203873:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203880:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.203883:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.203884:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.203890:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510757376:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.203891:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203900:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203908:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aed73445/0xf62499ab927a572c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.203912:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.203920:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.203923:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203930:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.203933:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203945:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.203946:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efc700/2, rc 0, transno 4295070742 00000100:00100000:1.0:1785220582.203954:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510754560:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.203956:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203963:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5725 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203976:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220582.203976:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:3.0:1785220582.203980:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.203983:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa62d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510757504:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.203987:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.203990:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.203997:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.204004:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.204006:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.204013:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a56e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.204028:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10b100/2, rc 0, transno 4295070743 00010000:00010000:3.0:1785220582.204032:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.204034:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.204035:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510751616:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.204036:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.204041:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.204051:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510757248 00000100:00100000:3.0:1785220582.204056:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510757248 00000100:00100000:1.0:1785220582.204058:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c682680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510757632:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.204079:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.204099:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510757376 00000100:00100000:3.0:1785220582.204103:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510757376 00000100:00100000:1.0:1785220582.204111:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510757504 00000100:00100000:1.0:1785220582.204114:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510757504 00000100:00100000:1.0:1785220582.204132:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510757632 00000100:00100000:3.0:1785220582.204134:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284692 00000100:00100000:1.0:1785220582.204134:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510757632 00000100:00100000:3.0:1785220582.204142:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+122:3652:x1871938510757248:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.204149:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284693 00000100:00100000:1.0:1785220582.204154:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+122:3652:x1871938510757376:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.204164:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29441 count 1 00002000:00080000:1.0:1785220582.204164:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29442 count 1 00010000:00010000:3.0:1785220582.204181:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.204182:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.204198:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000032352294/0xf62499ab92793308 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ed expref: 109 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.204198:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc4935dc/0xf62499ab92793316 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7fb expref: 109 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.204207:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7301:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.204207:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7302:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.204211:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7301:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.204211:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7302:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.204217:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.204217:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.204241:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000032352294/0xf62499ab92793308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ed expref: 109 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.204241:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cc4935dc/0xf62499ab92793316 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7fb expref: 109 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.204255:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000032352294/0xf62499ab92793308 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ed expref: 109 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.204255:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cc4935dc/0xf62499ab92793316 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7fb expref: 109 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.204267:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510757888:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.204267:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0000_UUID:14183:1871938510757760:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.205216:0:14193:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284694 00000100:00100000:2.0:1785220582.205231:0:14193:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+122:3655:x1871938510757504:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.205257:0:14193:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29443 count 1 00010000:00010000:2.0:1785220582.205275:0:14193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.205292:0:14193:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005b8f9c30/0xf62499ab92793324 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f809 expref: 109 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.205300:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7303:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.205303:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7303:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.205307:0:14193:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.205332:0:14193:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005b8f9c30/0xf62499ab92793324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f809 expref: 109 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.205342:0:14193:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005b8f9c30/0xf62499ab92793324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f809 expref: 109 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.205354:0:14193:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97add180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510758016:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.205518:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.205538:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284695 00000100:00100000:1.0:1785220582.205546:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+122:3655:x1871938510757632:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.205558:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29444 count 1 00010000:00010000:1.0:1785220582.205570:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.205583:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007b144ed0/0xf62499ab92793332 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f817 expref: 109 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.205590:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7304:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.205592:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7304:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.205596:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.205621:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007b144ed0/0xf62499ab92793332 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f817 expref: 109 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.205631:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007b144ed0/0xf62499ab92793332 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f817 expref: 109 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.205640:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510758144:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.205670:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.205691:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.205700:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.205729:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.205835:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510755712:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.205855:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.205894:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.205901:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.205924:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.205934:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.205939:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.205946:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.205950:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.205958:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.205974:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.205981:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a5709 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.205995:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.205997:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.205999:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.206036:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.206055:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1315:3655:x1871938510752384:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19519us (20285us total) trans 4295070166 rc 0/0 00000100:00100000:0.0:1785220582.206064:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284677 00000100:00100000:3.0:1785220582.206067:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.206123:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510755840:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.206140:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206159:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206168:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206176:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206183:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206199:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.206227:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.206232:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206239:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.206243:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206250:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206260:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206268:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5717 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.206276:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+123:3655:x1871938510753408:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19217us (19454us total) trans 4295070744 rc 0/0 00010000:00010000:3.0:1785220582.206281:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.206282:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.206283:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284679 00010000:00010000:3.0:1785220582.206284:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.206375:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986048 00000100:00100000:0.0:1785220582.206380:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986048 00010000:00010000:0.0:1785220582.206400:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000558ff3eb/0xf62499ab92791d3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f58c expref: 141 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.206425:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986176 00000100:00100000:0.0:1785220582.206428:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986176 00010000:00010000:0.0:1785220582.206439:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009f0200c4/0xf62499ab92791dc2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f59a expref: 142 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.206458:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100617 00000100:00100000:0.0:1785220582.206465:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce7bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+142:2407:x1871938519986048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.206474:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.206479:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000558ff3eb/0xf62499ab92791d3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f58c expref: 142 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206488:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000558ff3eb/0xf62499ab92791d3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f58c expref: 142 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206504:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000558ff3eb/0xf62499ab92791d3d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f58c expref: 142 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206515:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.206518:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.206523:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206531:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206688:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.206721:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce7bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+141:2407:x1871938519986048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 255us (684us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.206727:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100617 00000100:00100000:0.0:1785220582.206771:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100618 00000100:00100000:0.0:1785220582.206779:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+140:2406:x1871938519986176:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.206787:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.206792:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009f0200c4/0xf62499ab92791dc2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f59a expref: 140 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206802:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009f0200c4/0xf62499ab92791dc2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f59a expref: 140 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206815:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009f0200c4/0xf62499ab92791dc2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f59a expref: 140 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206881:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.206885:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.206889:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206897:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206913:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.206932:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+139:2406:x1871938519986176:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 154us (863us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.206938:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100618 00010000:00010000:0.0:1785220582.206957:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206969:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206970:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206978:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206980:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206987:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206989:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.206996:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.206999:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.207008:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.207175:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97ade300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1316:3652:x1871938510755328:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13535us (13818us total) trans 4295070167 rc 0/0 00000100:00100000:0.0:1785220582.207182:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284685 00010000:00010000:0.0:1785220582.207200:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.207205:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.207213:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.207216:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.207224:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.207228:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97ade680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1315:3652:x1871938510755200:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13862us (13921us total) trans 4295070168 rc 0/0 00000100:00100000:3.0:1785220582.207235:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284684 00010000:00010000:0.0:1785220582.207235:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.207242:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094b7a2d3/0xf62499ab927a5741 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.207252:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.207256:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.207257:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.207259:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.207260:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.207263:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.207267:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.207274:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.207284:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.207290:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a573a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.207304:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.207306:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.207307:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.207783:0:14193:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6a80 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.207800:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.207899:0:14189:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7656ce5/0xf62499ab927932de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7c3 expref: 109 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.207913:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510755072:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.207933:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.207942:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.207948:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.207956:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.207964:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.207982:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.207988:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.207993:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.208000:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.208002:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.208010:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.208022:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.208028:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5733 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.208044:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.208045:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.208047:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.208118:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97adf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+123:3654:x1871938510754688:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17320us (18462us total) trans 4295070745 rc 0/0 00000100:00100000:2.0:1785220582.208126:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284683 00000100:00100000:0.0:1785220582.208214:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510755968:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.208228:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.208298:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510756352:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.208316:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.208376:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510756096:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.208389:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.208502:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.208533:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510758272:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.208567:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.208597:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adca80/2, rc 0, transno 4295070167 00000100:00100000:1.0:1785220582.208608:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510755328:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.208653:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adc000/2, rc 0, transno 4295070168 00000100:00100000:1.0:1785220582.208661:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510755200:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.208717:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.208724:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510758400:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.208760:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.208809:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:13909:1871938510756864:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.208826:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.208933:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbe300/2, rc 0, transno 4295070744 00000100:00100000:1.0:1785220582.208944:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510753408:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.208983:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89fb0e00/2, rc 0, transno 4295070166 00000100:00100000:1.0:1785220582.208990:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89fb0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510752384:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.209021:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510758784:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.209046:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.209263:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510758656:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.209287:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.209334:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510758272 00000100:00100000:0.0:1785220582.209340:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510758272 00000100:00100000:0.0:1785220582.209378:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510758400 00000100:00100000:0.0:1785220582.209381:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510758400 00000100:00100000:0.0:1785220582.209396:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510758784 00000100:00100000:0.0:1785220582.209398:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510758784 00000100:00100000:0.0:1785220582.209416:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284696 00000100:00100000:0.0:1785220582.209423:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1317:3652:x1871938510758272:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.209448:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29403 count 1 00010000:00010000:0.0:1785220582.209468:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209484:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a43c2694/0xf62499ab927920c4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e0 expref: 138 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209491:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72db:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.209496:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.209500:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209524:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a43c2694/0xf62499ab927920c4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e0 expref: 138 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209536:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a43c2694/0xf62499ab927920c4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e0 expref: 138 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.209546:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d313480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510758912:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.209592:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26240 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.209608:0:14169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284697 00000100:00100000:0.0:1785220582.209616:0:14169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1317:3652:x1871938510758400:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.209629:0:14169:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29404 count 1 00010000:00010000:0.0:1785220582.209640:0:14169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209652:0:14169:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002cf9ef2b/0xf62499ab9279216c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5ee expref: 138 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209659:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72dc:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.209662:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.209665:0:14169:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209684:0:14169:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002cf9ef2b/0xf62499ab9279216c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5ee expref: 138 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209693:0:14169:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002cf9ef2b/0xf62499ab9279216c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5ee expref: 138 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.209702:0:14169:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510759040:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.209725:0:14169:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.209740:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284698 00000100:00100000:0.0:1785220582.209746:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1317:3652:x1871938510758784:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.209772:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29406 count 1 00010000:00010000:0.0:1785220582.209784:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209795:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005efdecab/0xf62499ab92792253 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f60a expref: 138 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209807:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72de:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.209810:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.209813:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209831:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005efdecab/0xf62499ab92792253 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f60a expref: 138 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.209840:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005efdecab/0xf62499ab92792253 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f60a expref: 138 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.209850:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510759168:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.209879:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d86d26d80 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.210205:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89effb80/2, rc 0, transno 4295070745 00000100:00100000:0.0:1785220582.210224:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89effb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510754688:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.210237:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510758528:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.210364:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220582.210374:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97adf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510759296:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.210798:0:14161:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510758656 00000100:00100000:3.0:1785220582.210804:0:14161:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510758656 00000100:00100000:0.0:1785220582.210815:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.210945:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510756992:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.210965:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211002:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.211021:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.211044:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.211059:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986432 00000100:00100000:0.0:1785220582.211063:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986432 00000100:00100000:3.0:1785220582.211205:0:14161:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510759296 00000100:00100000:3.0:1785220582.211209:0:14161:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510759296 00000100:00100000:3.0:1785220582.211234:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284699 00000100:00100000:3.0:1785220582.211242:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+124:3655:x1871938510758656:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.211258:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006fe71057/0xf62499ab92791f04 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5b6 expref: 139 pid: 14187 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220582.211261:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29445 count 1 00010000:00010000:3.0:1785220582.211279:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211281:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986304 00000100:00100000:0.0:1785220582.211284:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986304 00010000:00010000:3.0:1785220582.211294:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000be5db5f5/0xf62499ab92793340 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f825 expref: 108 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211298:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003f264abb/0xf62499ab92791e5c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a8 expref: 140 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211303:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7305:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.211306:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7305:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.211310:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211312:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986688 00000100:00100000:0.0:1785220582.211314:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986688 00010000:00010000:0.0:1785220582.211327:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000033044c95/0xf62499ab927932ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d1 expref: 109 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211334:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000be5db5f5/0xf62499ab92793340 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f825 expref: 109 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211344:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100619 00010000:00010000:3.0:1785220582.211345:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000be5db5f5/0xf62499ab92793340 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f825 expref: 109 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211350:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+140:2406:x1871938519986432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.211358:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510759424:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.211358:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.211363:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006fe71057/0xf62499ab92791f04 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5b6 expref: 140 pid: 14187 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211373:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006fe71057/0xf62499ab92791f04 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5b6 expref: 140 pid: 14187 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211393:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006fe71057/0xf62499ab92791f04 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5b6 expref: 140 pid: 14187 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.211404:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.211408:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.211411:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.211416:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.211419:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284700 00000100:00100000:3.0:1785220582.211426:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+124:3652:x1871938510759296:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.211426:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220582.211437:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29446 count 1 00010000:00010000:0.0:1785220582.211445:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.211448:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211458:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003c3a72e5/0xf62499ab9279334e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f833 expref: 109 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211465:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7306:0x0].0x0 since=17s v=(0 0) 00000100:00100000:0.0:1785220582.211466:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+139:2406:x1871938519986432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (463us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.211468:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7306:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.211472:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211473:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100619 00000100:00100000:0.0:1785220582.211488:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100620 00010000:00010000:3.0:1785220582.211489:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003c3a72e5/0xf62499ab9279334e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f833 expref: 109 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211494:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+138:2407:x1871938519986304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.211498:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003c3a72e5/0xf62499ab9279334e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f833 expref: 109 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211502:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.211506:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003f264abb/0xf62499ab92791e5c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a8 expref: 138 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.211508:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97843b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510759552:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.211516:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003f264abb/0xf62499ab92791e5c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a8 expref: 138 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.211529:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920c00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.211533:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003f264abb/0xf62499ab92791e5c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5a8 expref: 138 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211543:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.211546:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.211550:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.211551:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100621 00000100:00100000:3.0:1785220582.211558:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+109:2406:x1871938519986688:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.211558:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211568:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.211573:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000033044c95/0xf62499ab927932ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d1 expref: 109 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211574:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.211582:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000033044c95/0xf62499ab927932ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d1 expref: 109 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.211589:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+137:2407:x1871938519986304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.211595:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100620 00010000:00010000:3.0:1785220582.211596:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033044c95/0xf62499ab927932ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d1 expref: 109 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211606:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.211620:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211623:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x72ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.211628:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211629:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211636:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211640:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211648:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.211654:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.211661:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.211672:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+108:2406:x1871938519986688:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (627us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.211678:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100621 00000100:00100000:0.0:1785220582.211815:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97add880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+1318:3654:x1871938510755584:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17882us (18331us total) trans 4295070169 rc 0/0 00000100:00100000:0.0:1785220582.211929:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284687 00010000:00010000:0.0:1785220582.211952:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211961:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211968:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211974:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.211995:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.212067:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97adc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1318:3654:x1871938510755456:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18271us (18620us total) trans 4295070170 rc 0/0 00000100:00100000:0.0:1785220582.212074:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284686 00010000:00010000:0.0:1785220582.212118:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212128:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212141:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212147:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212160:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.212254:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+125:3652:x1871938510756480:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13041us (13475us total) trans 4295070746 rc 0/0 00000100:00100000:0.0:1785220582.212260:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284689 00010000:00010000:0.0:1785220582.212276:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220582.212283:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212291:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.212295:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212304:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212316:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212324:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000722c2c8e/0xf62499ab927a574f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212341:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.212344:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212351:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.212354:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212363:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212373:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212380:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5748 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212392:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.212396:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212402:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.212406:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212419:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212430:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212437:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3a898ca/0xf62499ab927a575d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212453:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.212455:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.212457:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.212491:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083ea00/2, rc 0, transno 4295070746 00010000:00010000:3.0:1785220582.212502:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.212502:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510756480:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.212541:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.212563:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986560 00000100:00100000:0.0:1785220582.212567:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986560 00010000:00010000:0.0:1785220582.212585:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dcd58a22/0xf62499ab92791f9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5c4 expref: 137 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.212608:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100622 00000100:00100000:0.0:1785220582.212616:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+137:2407:x1871938519986560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.212625:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.212630:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dcd58a22/0xf62499ab92791f9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5c4 expref: 137 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212641:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dcd58a22/0xf62499ab92791f9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5c4 expref: 137 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212654:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dcd58a22/0xf62499ab92791f9e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5c4 expref: 137 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212665:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.212667:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72d9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.212672:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212680:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.212708:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.212730:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+136:2407:x1871938519986560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.212737:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100622 00000100:00100000:0.0:1785220582.212810:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.213056:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986816 00000100:00100000:0.0:1785220582.213060:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986816 00010000:00010000:0.0:1785220582.213076:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000a7c65e2/0xf62499ab9279201c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d2 expref: 136 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.213123:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100623 00000100:00100000:0.0:1785220582.213131:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+136:2407:x1871938519986816:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.213139:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.213144:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000a7c65e2/0xf62499ab9279201c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d2 expref: 136 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213154:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000a7c65e2/0xf62499ab9279201c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d2 expref: 136 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213169:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a7c65e2/0xf62499ab9279201c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5d2 expref: 136 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213178:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.213181:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72da:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.213184:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213191:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213208:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.213227:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+135:2407:x1871938519986816:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (416us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.213233:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100623 00010000:00010000:3.0:1785220582.213245:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213250:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.213261:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213264:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.213270:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213271:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.213279:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213279:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.213291:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.213459:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+1318:3655:x1871938510756736:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14035us (14583us total) trans 4295070171 rc 0/0 00000100:00100000:0.0:1785220582.213465:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284690 00000100:00100000:0.0:1785220582.213534:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0000_UUID:14160:1871938510757120:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.213552:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.214160:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0000_UUID:14183:1871938510757760:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.214179:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.214211:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbe680/2, rc 0, transno 4295070171 00000100:00100000:1.0:1785220582.214221:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510756736:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.214268:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.214281:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510759680:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.214312:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.214615:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.214622:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214630:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.214634:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214643:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214659:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214672:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5764 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214694:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.214698:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214706:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.214709:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214716:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214726:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.214733:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.214735:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.214737:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.214893:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510759680 00000100:00100000:0.0:1785220582.214899:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510759680 00000100:00100000:0.0:1785220582.214943:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284701 00000100:00100000:0.0:1785220582.214952:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1318:3655:x1871938510759680:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.214975:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29407 count 1 00010000:00010000:0.0:1785220582.214995:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.215015:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006ba586b8/0xf62499ab927922ca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f618 expref: 134 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.215023:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72df:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.215028:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.215032:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.215060:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006ba586b8/0xf62499ab927922ca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f618 expref: 134 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.215073:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006ba586b8/0xf62499ab927922ca lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f618 expref: 134 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.215111:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c682300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510759808:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.215165:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.215526:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220582.215534:0:14207:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5756 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72d9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.215687:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.215766:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.215853:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97addf80/2, rc 0, transno 4295070169 00000100:00100000:2.0:1785220582.215865:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97addf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510755584:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.215900:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1319:3653:x1871938510756224:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20903us (20976us total) trans 4295070172 rc 0/0 00000100:00100000:2.0:1785220582.215900:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510760064:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.215907:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284688 00000100:00100000:2.0:1785220582.215926:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.215965:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510759936:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.216293:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97add500/2, rc 0, transno 4295070170 00000100:00100000:2.0:1785220582.216302:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97add500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510755456:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.216338:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.216346:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d96667b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510760192:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.216371:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.216406:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510758528 00000100:00100000:0.0:1785220582.216406:0:14175:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510760064 00000100:00100000:3.0:1785220582.216411:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510758528 00000100:00100000:0.0:1785220582.216411:0:14175:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510760064 00000100:00100000:0.0:1785220582.216440:0:14175:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284702 00000100:00100000:3.0:1785220582.216445:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284703 00000100:00100000:0.0:1785220582.216447:0:14175:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1320:3652:x1871938510760064:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.216450:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1320:3654:x1871938510758528:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.216463:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29405 count 1 00002000:00080000:0.0:1785220582.216463:0:14175:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29408 count 1 00010000:00010000:3.0:1785220582.216480:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.216480:0:14175:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.216496:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000046516212/0xf62499ab927921a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5fc expref: 134 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.216496:0:14175:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c5f5bc7b/0xf62499ab92792387 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f626 expref: 134 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.216505:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72dd:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.216505:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e0:0x0].0x0 since=19s v=(0 0) 00010000:00010000:3.0:1785220582.216509:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.216510:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.216515:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.216515:0:14175:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.216538:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000046516212/0xf62499ab927921a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5fc expref: 134 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.216538:0:14175:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c5f5bc7b/0xf62499ab92792387 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f626 expref: 134 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.216550:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000046516212/0xf62499ab927921a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5fc expref: 134 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.216550:0:14175:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c5f5bc7b/0xf62499ab92792387 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f626 expref: 134 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.216563:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510760320:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.216563:0:14175:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c680000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510760448:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.216606:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.216672:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510760192 00000100:00100000:1.0:1785220582.216677:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510760192 00000100:00100000:1.0:1785220582.216704:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284704 00000100:00100000:1.0:1785220582.216710:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d96664e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1321:3652:x1871938510760192:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.216722:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29409 count 1 00010000:00010000:1.0:1785220582.216735:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.216746:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000099757d0f/0xf62499ab9279246e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f634 expref: 134 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.216763:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e1:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.216766:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.216770:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.216789:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000099757d0f/0xf62499ab9279246e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f634 expref: 134 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.216797:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000099757d0f/0xf62499ab9279246e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f634 expref: 134 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.216806:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510760576:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.217034:0:14175:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.217055:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.217081:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510759936 00010000:00010000:3.0:1785220582.217404:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.217521:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0df80/2, rc 0, transno 4295070172 00000100:00100000:0.0:1785220582.217531:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510759936 00000100:00100000:3.0:1785220582.217533:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510756224:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.217562:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284705 00000100:00100000:0.0:1785220582.217569:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+125:3655:x1871938510759936:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.217583:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29447 count 1 00010000:00010000:0.0:1785220582.217599:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.217618:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004cebbaf5/0xf62499ab9279335c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f841 expref: 107 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.217625:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7307:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.217628:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7307:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.217632:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.217653:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004cebbaf5/0xf62499ab9279335c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f841 expref: 107 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.217664:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004cebbaf5/0xf62499ab9279335c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f841 expref: 107 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.217676:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c680a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510760832:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.217709:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.217729:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.217737:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510760704:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.217779:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.217983:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.217998:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510760704 00000100:00100000:1.0:1785220582.218002:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510760704 00000100:00100000:1.0:1785220582.218080:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284706 00000100:00100000:1.0:1785220582.218225:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d311180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1322:3655:x1871938510760704:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.218238:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29410 count 1 00010000:00010000:1.0:1785220582.218253:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.218270:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e8c4afe7/0xf62499ab9279254e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f642 expref: 134 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.218276:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e2:0x0].0x0 since=19s v=(0 0) 00010000:00010000:1.0:1785220582.218279:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.218282:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.218303:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e8c4afe7/0xf62499ab9279254e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f642 expref: 134 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.218312:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e8c4afe7/0xf62499ab9279254e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f642 expref: 134 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.218322:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510760960:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.218351:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.220362:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.220425:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.220472:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510757888:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.220499:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.220513:0:14193:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97add180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0000_UUID:14193:1871938510758016:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.220527:0:14193:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.220557:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.220700:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.220737:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987072 00000100:00100000:1.0:1785220582.220743:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987072 00010000:00010000:1.0:1785220582.220786:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4935dc/0xf62499ab92793316 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7fb expref: 108 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.220818:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519986944 00000100:00100000:1.0:1785220582.220821:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519986944 00010000:00010000:1.0:1785220582.220836:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ff19f5e9/0xf62499ab927932fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7df expref: 109 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.220860:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987200 00000100:00100000:1.0:1785220582.220862:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987200 00010000:00010000:1.0:1785220582.220874:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000032352294/0xf62499ab92793308 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ed expref: 110 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.220893:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100624 00000100:00100000:1.0:1785220582.220902:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+110:2407:x1871938519987072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.220914:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.220919:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cc4935dc/0xf62499ab92793316 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7fb expref: 110 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.220930:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cc4935dc/0xf62499ab92793316 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7fb expref: 110 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.220952:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4935dc/0xf62499ab92793316 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7fb expref: 110 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.220966:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7302:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.220970:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7302:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.220976:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.220986:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.220995:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100625 00000100:00100000:3.0:1785220582.221004:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d96666a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+109:2407:x1871938519986944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.221012:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.221012:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.221017:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ff19f5e9/0xf62499ab927932fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7df expref: 109 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221026:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ff19f5e9/0xf62499ab927932fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7df expref: 109 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221037:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff19f5e9/0xf62499ab927932fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7df expref: 109 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221047:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7300:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.221050:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7300:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.221053:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+108:2407:x1871938519987072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 152us (699us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.221055:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.221060:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100624 00010000:00010000:3.0:1785220582.221063:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221078:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.221131:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d96666a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+107:2407:x1871938519986944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 128us (704us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.221138:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100625 00000100:00100000:3.0:1785220582.221152:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100626 00000100:00100000:3.0:1785220582.221159:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97add180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2406:x1871938519987200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.221167:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.221172:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000032352294/0xf62499ab92793308 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ed expref: 106 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221186:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000032352294/0xf62499ab92793308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ed expref: 106 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221190:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221199:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032352294/0xf62499ab92793308 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ed expref: 106 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221199:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221210:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7301:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.221210:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221213:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7301:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.221217:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221218:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221225:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221235:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.221238:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.221248:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221255:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221262:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221269:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.221281:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.221424:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+127:3652:x1871938510757376:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17268us (17509us total) trans 4295070748 rc 0/0 00000100:00100000:1.0:1785220582.221432:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284693 00010000:00010000:1.0:1785220582.221456:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.221462:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221470:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.221475:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221484:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221497:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221504:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5779 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221520:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.221523:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221530:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.221533:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221540:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221549:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221556:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a5772 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.221569:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.221571:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.221573:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.221579:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.221604:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0000_UUID:14252:1871938510758144:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.221619:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.221687:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+126:3652:x1871938510757248:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17545us (17873us total) trans 4295070747 rc 0/0 00000100:00100000:3.0:1785220582.221695:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284692 00000100:00100000:3.0:1785220582.221725:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97add180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+105:2406:x1871938519987200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 566us (1167us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.221731:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100626 00000004:00080000:2.0:1785220582.221800:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa63800/2, rc 0, transno 4295070748 00000100:00100000:2.0:1785220582.221812:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510757376:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.221873:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083ce00/2, rc 0, transno 4295070747 00000100:00100000:2.0:1785220582.221881:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510757248:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.221920:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93760700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510761216:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.221951:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.221978:0:14196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510761216 00000100:00100000:2.0:1785220582.221982:0:14196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510761216 00000100:00100000:2.0:1785220582.222014:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284707 00000100:00100000:2.0:1785220582.222022:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93760a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+126:3652:x1871938510761216:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.222041:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29449 count 1 00010000:00010000:2.0:1785220582.222060:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222074:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f6b09c95/0xf62499ab92793378 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f85d expref: 104 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222080:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7309:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.222117:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7309:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.222123:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222146:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f6b09c95/0xf62499ab92793378 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f85d expref: 104 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222159:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f6b09c95/0xf62499ab92793378 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f85d expref: 104 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.222171:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93761180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510761344:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.222210:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.222232:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.222240:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510761088:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.222265:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.222287:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510761088 00000100:00100000:2.0:1785220582.222292:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510761088 00000100:00100000:2.0:1785220582.222317:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284708 00000100:00100000:2.0:1785220582.222323:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+127:3654:x1871938510761088:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.222335:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29448 count 1 00010000:00010000:2.0:1785220582.222347:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222358:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000de3a5785/0xf62499ab9279336a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f84f expref: 104 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222365:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7308:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.222368:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7308:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.222372:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222391:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000de3a5785/0xf62499ab9279336a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f84f expref: 104 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.222401:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000de3a5785/0xf62499ab9279336a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f84f expref: 104 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.222410:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93761500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510761472:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.222431:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d63c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.222493:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.222680:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987584 00000100:00100000:0.0:1785220582.222684:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987584 00010000:00010000:0.0:1785220582.222709:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a43c2694/0xf62499ab927920c4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e0 expref: 135 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.222732:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987456 00000100:00100000:0.0:1785220582.222735:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987456 00010000:00010000:0.0:1785220582.222760:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007b144ed0/0xf62499ab92793332 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f817 expref: 105 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.222781:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100627 00000100:00100000:0.0:1785220582.222788:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+135:2407:x1871938519987584:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.222797:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.222803:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a43c2694/0xf62499ab927920c4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e0 expref: 135 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222813:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a43c2694/0xf62499ab927920c4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e0 expref: 135 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222829:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.222832:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72db:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.222838:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222846:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222862:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.222885:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+135:2407:x1871938519987584:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (2185us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.222891:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100627 00000100:00100000:0.0:1785220582.222905:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100628 00000100:00100000:0.0:1785220582.222912:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+105:2407:x1871938519987456:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.222920:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.222924:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007b144ed0/0xf62499ab92793332 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f817 expref: 105 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222934:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007b144ed0/0xf62499ab92793332 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f817 expref: 105 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222946:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b144ed0/0xf62499ab92793332 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f817 expref: 105 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222957:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7304:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.222960:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7304:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.222963:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222971:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.222989:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.223008:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+104:2407:x1871938519987456:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (515us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.223015:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100628 00010000:00010000:0.0:1785220582.223050:0:14209:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a43c2694/0xf62499ab927920c4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5e0 expref: 134 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.223067:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d313480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0001_UUID:14209:1871938510758912:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.223111:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223122:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223129:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223138:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223146:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223166:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.223305:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+1323:3652:x1871938510758272:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13881us (14736us total) trans 4295070173 rc 0/0 00000100:00100000:0.0:1785220582.223312:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284696 00000100:00100000:0.0:1785220582.223342:0:14169:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510759040:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.223355:0:14169:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223379:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223386:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223393:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223400:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223416:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.223498:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+128:3655:x1871938510756608:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23474us (24530us total) trans 4295070749 rc 0/0 00000100:00100000:0.0:1785220582.223505:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284691 00010000:00010000:0.0:1785220582.223530:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.223537:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223545:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.223550:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223558:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223570:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223577:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a578e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72db:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223595:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.223599:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223605:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.223609:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223616:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223626:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223632:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e45a31b4/0xf62499ab927a576b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.223646:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.223648:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.223650:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.223677:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083f480/2, rc 0, transno 4295070749 00000100:00100000:0.0:1785220582.223689:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510756608:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.223919:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510761600:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.223952:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.224383:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.224631:0:13874:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510761600 00000100:00100000:1.0:1785220582.224638:0:13874:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510761600 00000100:00100000:1.0:1785220582.224674:0:13874:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284709 00000100:00100000:1.0:1785220582.224682:0:13874:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+128:3653:x1871938510761600:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.224705:0:13874:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29450 count 1 00010000:00010000:1.0:1785220582.224727:0:13874:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224744:0:13874:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f7bce6c9/0xf62499ab92793386 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f86b expref: 103 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224763:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x730a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.224767:0:13874:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.224771:0:13874:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224798:0:13874:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f7bce6c9/0xf62499ab92793386 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f86b expref: 103 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224812:0:13874:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f7bce6c9/0xf62499ab92793386 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f86b expref: 103 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.224824:0:13874:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510761728:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.224874:0:13874:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.224895:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224904:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224913:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224922:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.224940:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.225074:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+129:3655:x1871938510757632:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19528us (20993us total) trans 4295070750 rc 0/0 00000100:00100000:1.0:1785220582.225081:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284695 00000004:00080000:2.0:1785220582.225320:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d312a00/2, rc 0, transno 4295070173 00000100:00100000:2.0:1785220582.225333:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510758272:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.225385:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93762680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510761856:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.225517:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c682680/2, rc 0, transno 4295070750 00000100:00100000:2.0:1785220582.225566:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.225621:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.225648:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.225696:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.225715:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.225760:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.225778:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987712 00000100:00100000:2.0:1785220582.225782:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987712 00010000:00010000:2.0:1785220582.225806:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002cf9ef2b/0xf62499ab9279216c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5ee expref: 134 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.225828:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987328 00000100:00100000:2.0:1785220582.225831:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987328 00010000:00010000:2.0:1785220582.225846:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005b8f9c30/0xf62499ab92793324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f809 expref: 104 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.225863:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987840 00000100:00100000:2.0:1785220582.225866:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987840 00010000:00010000:2.0:1785220582.225879:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000be5db5f5/0xf62499ab92793340 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f825 expref: 105 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.225895:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519987968 00000100:00100000:2.0:1785220582.225897:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519987968 00010000:00010000:2.0:1785220582.225909:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005efdecab/0xf62499ab92792253 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f60a expref: 135 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.225925:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988096 00000100:00100000:2.0:1785220582.225928:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988096 00010000:00010000:2.0:1785220582.225940:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003c3a72e5/0xf62499ab9279334e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f833 expref: 106 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.225960:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100629 00000100:00100000:2.0:1785220582.225967:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93762a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+135:2407:x1871938519987712:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.225977:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.225982:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002cf9ef2b/0xf62499ab9279216c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5ee expref: 135 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.225993:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002cf9ef2b/0xf62499ab9279216c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5ee expref: 135 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226013:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002cf9ef2b/0xf62499ab9279216c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5ee expref: 135 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226026:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.226030:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72dc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.226036:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226044:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226066:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.226189:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93762a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+134:2407:x1871938519987712:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 222us (568us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.226197:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100629 00000100:00100000:2.0:1785220582.226212:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100630 00000100:00100000:2.0:1785220582.226219:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93760e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2406:x1871938519987328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.226227:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.226232:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005b8f9c30/0xf62499ab92793324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f809 expref: 106 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226241:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005b8f9c30/0xf62499ab92793324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f809 expref: 106 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.226245:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97843b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13818:1871938510759552:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.226254:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b8f9c30/0xf62499ab92793324 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f809 expref: 106 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226264:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7303:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.226265:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226267:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7303:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.226272:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226279:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.226281:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.226286:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.226293:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.226295:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.226298:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.226306:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.226312:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93760e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+105:2406:x1871938519987328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (664us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.226319:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100630 00010000:00010000:3.0:1785220582.226324:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.226331:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100631 00010000:00010000:3.0:1785220582.226332:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cb21e0b/0xf62499ab927a5787 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.226337:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c307450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+104:2407:x1871938519987840:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.226343:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.226347:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000be5db5f5/0xf62499ab92793340 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f825 expref: 104 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.226349:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.226350:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.226352:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.226357:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000be5db5f5/0xf62499ab92793340 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f825 expref: 104 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.226370:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97842a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510759424:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.226370:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be5db5f5/0xf62499ab92793340 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f825 expref: 104 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226380:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7305:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.226381:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.226383:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7305:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.226388:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.226436:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c682680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510757632:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.226448:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.226466:0:14185:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510761856 00000100:00100000:0.0:1785220582.226469:0:14185:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510761856 00000100:00100000:0.0:1785220582.226528:0:14185:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284710 00000100:00100000:0.0:1785220582.226538:0:14185:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea6935450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1323:3654:x1871938510761856:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.226557:0:14185:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29411 count 1 00010000:00010000:0.0:1785220582.226576:0:14185:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226587:0:14185:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009c39f102/0xf62499ab927925cc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f650 expref: 133 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226595:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e3:0x0].0x0 since=19s v=(0 0) 00010000:00010000:0.0:1785220582.226597:0:14185:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.226601:0:14185:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226622:0:14185:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009c39f102/0xf62499ab927925cc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f650 expref: 133 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226632:0:14185:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009c39f102/0xf62499ab927925cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f650 expref: 133 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.226649:0:14185:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d313480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510761984:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.226687:0:14185:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.226710:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510759168:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.226722:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226736:0:14169:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226744:0:14169:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226763:0:14169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226770:0:14169:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226784:0:14169:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.226914:0:14169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1324:3652:x1871938510758400:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17299us (18164us total) trans 4295070174 rc 0/0 00000100:00100000:0.0:1785220582.226921:0:14169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284697 00000100:00100000:0.0:1785220582.226940:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100632 00000100:00100000:0.0:1785220582.226946:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93761f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2407:x1871938519987968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.226953:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.226958:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005efdecab/0xf62499ab92792253 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f60a expref: 133 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226967:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005efdecab/0xf62499ab92792253 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f60a expref: 133 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226980:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005efdecab/0xf62499ab92792253 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f60a expref: 133 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.226989:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.226992:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72de:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.226996:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227003:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227019:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.227030:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93761f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+132:2407:x1871938519987968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (1315us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.227036:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100632 00000100:00100000:0.0:1785220582.227047:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100633 00000100:00100000:0.0:1785220582.227052:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+103:2406:x1871938519988096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.227059:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.227067:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003c3a72e5/0xf62499ab9279334e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f833 expref: 103 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227076:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003c3a72e5/0xf62499ab9279334e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f833 expref: 103 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227114:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c3a72e5/0xf62499ab9279334e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f833 expref: 103 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227124:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7306:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.227127:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7306:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.227131:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227138:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227157:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.227171:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+102:2406:x1871938519988096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 119us (1420us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.227177:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100633 00010000:00010000:0.0:1785220582.227197:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227207:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227223:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227229:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227240:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.227303:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1324:3652:x1871938510758784:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17557us (18256us total) trans 4295070175 rc 0/0 00000100:00100000:0.0:1785220582.227306:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284698 00010000:00010000:0.0:1785220582.227318:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.227321:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227327:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.227329:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227334:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227341:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227345:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a5795 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72dc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227356:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.227358:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227363:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.227364:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227368:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227373:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227376:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a579c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72de:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227383:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.227384:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.227384:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.227472:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c682300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510759808:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.227482:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.227510:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.227522:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988224 00000100:00100000:0.0:1785220582.227524:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988224 00010000:00010000:0.0:1785220582.227535:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006ba586b8/0xf62499ab927922ca lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f618 expref: 132 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.227549:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100634 00000100:00100000:0.0:1785220582.227552:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+132:2406:x1871938519988224:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.227557:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.227559:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006ba586b8/0xf62499ab927922ca lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f618 expref: 132 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227565:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006ba586b8/0xf62499ab927922ca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f618 expref: 132 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227574:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006ba586b8/0xf62499ab927922ca lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f618 expref: 132 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227581:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.227583:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72df:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.227585:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227590:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227599:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.227609:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+131:2406:x1871938519988224:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 56us (98us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.227614:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100634 00010000:00010000:0.0:1785220582.227632:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227639:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227647:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227655:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.227667:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.227723:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1324:3655:x1871938510759680:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12774us (13411us total) trans 4295070176 rc 0/0 00000100:00100000:0.0:1785220582.227728:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284701 00000004:00080000:0.0:1785220582.227761:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d998b8e00/2, rc 0, transno 4295070176 00000100:00100000:0.0:1785220582.227772:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510759680:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.227863:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1785220582.227866:0:14175:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c680000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510760448:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.227869:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227876:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.227878:0:14175:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227880:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227888:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227899:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227907:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e23c80f/0xf62499ab927a57b1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72df:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227921:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.227922:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.227924:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.227932:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227939:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227946:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227952:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.227964:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.228022:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c680a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0000_UUID:14227:1871938510760832:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.228035:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.228047:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+129:3652:x1871938510759296:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16621us (17232us total) trans 4295070751 rc 0/0 00000100:00100000:3.0:1785220582.228054:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284700 00010000:00010000:3.0:1785220582.228073:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.228077:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228108:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.228111:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228131:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.228140:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.228143:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228151:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a57aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228166:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.228166:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.228167:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.228169:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.228188:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510760320:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.228189:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.228201:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.228215:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.228220:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988480 00000100:00100000:3.0:1785220582.228224:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988480 00000100:00100000:0.0:1785220582.228227:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988736 00000100:00100000:0.0:1785220582.228229:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988736 00010000:00010000:3.0:1785220582.228238:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c5f5bc7b/0xf62499ab92792387 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f626 expref: 132 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228240:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000099757d0f/0xf62499ab9279246e lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f634 expref: 132 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.228256:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988352 00000100:00100000:0.0:1785220582.228258:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988352 00000100:00100000:3.0:1785220582.228259:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988608 00000100:00100000:3.0:1785220582.228261:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988608 00010000:00010000:0.0:1785220582.228268:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000046516212/0xf62499ab927921a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5fc expref: 133 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228270:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004cebbaf5/0xf62499ab9279335c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f841 expref: 102 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.228284:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100635 00000100:00100000:3.0:1785220582.228288:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100636 00000100:00100000:0.0:1785220582.228290:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2406:x1871938519988736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.228293:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+133:2406:x1871938519988480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.228298:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.228298:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.228303:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c5f5bc7b/0xf62499ab92792387 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f626 expref: 133 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228303:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000099757d0f/0xf62499ab9279246e lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f634 expref: 133 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228315:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c5f5bc7b/0xf62499ab92792387 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f626 expref: 133 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228315:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000099757d0f/0xf62499ab9279246e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f634 expref: 133 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228328:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c5f5bc7b/0xf62499ab92792387 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f626 expref: 133 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228331:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.228335:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.228338:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.228339:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228341:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.228345:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228347:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228353:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228359:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.228371:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.228377:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+132:2406:x1871938519988736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.228383:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100635 00000100:00100000:3.0:1785220582.228385:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+132:2406:x1871938519988480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.228391:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100636 00000100:00100000:0.0:1785220582.228395:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100637 00000100:00100000:3.0:1785220582.228401:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100638 00000100:00100000:0.0:1785220582.228401:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+130:2406:x1871938519988352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.228406:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c307c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+102:2406:x1871938519988608:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.228408:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.228412:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.228413:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000046516212/0xf62499ab927921a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5fc expref: 130 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228416:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004cebbaf5/0xf62499ab9279335c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f841 expref: 102 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228423:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000046516212/0xf62499ab927921a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5fc expref: 130 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228426:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004cebbaf5/0xf62499ab9279335c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f841 expref: 102 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228435:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000046516212/0xf62499ab927921a4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f5fc expref: 130 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228436:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cebbaf5/0xf62499ab9279335c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f841 expref: 102 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228446:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7307:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.228449:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7307:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.228449:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.228453:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228453:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72dd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.228456:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228461:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228464:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.228480:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.228481:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.228673:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+129:2406:x1871938519988352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 272us (482us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.228680:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100637 00010000:00010000:1.0:1785220582.228683:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.228702:0:14175:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.228703:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510760960:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.228710:0:14175:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228718:0:14175:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.228719:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228726:0:14175:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228739:0:14175:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.228744:0:14194:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000099757d0f/0xf62499ab9279246e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f634 expref: 128 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.228765:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510760576:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.228776:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.228784:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.228791:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.228798:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.228805:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.228836:0:14175:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1324:3652:x1871938510760064:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12389us (12909us total) trans 4295070177 rc 0/0 00000100:00100000:0.0:1785220582.228843:0:14175:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284702 00010000:00010000:0.0:1785220582.228862:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228870:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.228878:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.228879:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c307c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+101:2406:x1871938519988608:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (279us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.228886:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.228887:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100638 00010000:00010000:1.0:1785220582.228934:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220582.229004:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d96664e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1324:3652:x1871938510760192:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12294us (12633us total) trans 4295070178 rc 0/0 00000100:00100000:1.0:1785220582.229010:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284704 00010000:00010000:1.0:1785220582.229266:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.229275:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229308:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.229329:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c307450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+100:2407:x1871938519987840:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2992us (3632us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.229336:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100631 00010000:00010000:2.0:1785220582.229374:0:14193:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229387:0:14193:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229395:0:14193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229401:0:14193:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.229425:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.229432:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229476:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00010000:00010000:2.0:1785220582.229511:0:14193:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:0.0:1785220582.229566:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+130:3655:x1871938510759936:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11997us (12510us total) trans 4295070752 rc 0/0 00000100:00100000:0.0:1785220582.229573:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284705 00000100:00100000:2.0:1785220582.229579:0:14193:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+129:3655:x1871938510757504:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24355us (25574us total) trans 4295070753 rc 0/0 00000100:00100000:2.0:1785220582.229585:0:14193:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284694 00010000:00010000:0.0:1785220582.229589:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:0.0:1785220582.229595:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229598:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220582.229602:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229603:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.229606:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229620:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229621:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.229624:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229631:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229634:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229640:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229642:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bd049eb/0xf62499ab927a57bf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229648:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a57c6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229659:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.229660:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.229663:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229663:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229670:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.229670:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.229674:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229675:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229683:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229683:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229693:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229693:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229701:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5780 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229701:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe56d351/0xf62499ab927a57cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229712:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.229715:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229717:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.229719:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.229721:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.229722:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.229725:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229731:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.229736:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510762112:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.229740:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229746:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.229747:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.229760:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.229769:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229778:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.229779:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.229785:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229792:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.229810:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1785220582.229812:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbdf80/2, rc 0, transno 4295070752 00000100:00100000:0.0:1785220582.229820:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510759936:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.229827:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.229831:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229838:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.229841:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229848:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229858:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229865:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a57a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.229878:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.229880:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.229882:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.229925:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.229935:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510762240:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.229958:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.229982:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510762112 00000100:00100000:0.0:1785220582.229987:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510762112 00000100:00100000:2.0:1785220582.229989:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+129:3655:x1871938510758656:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18747us (20701us total) trans 4295070754 rc 0/0 00000100:00100000:2.0:1785220582.229998:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284699 00000100:00100000:0.0:1785220582.230012:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510762240 00000100:00100000:0.0:1785220582.230014:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510762240 00000100:00100000:2.0:1785220582.230015:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284711 00000100:00100000:2.0:1785220582.230022:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+130:3655:x1871938510762112:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.230028:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284712 00000100:00100000:0.0:1785220582.230033:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+130:3653:x1871938510762240:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.230042:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29451 count 1 00002000:00080000:0.0:1785220582.230042:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29452 count 1 00010000:00010000:2.0:1785220582.230059:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230059:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.230074:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c7bc1f10/0xf62499ab92793394 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f879 expref: 99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230074:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000020ea6998/0xf62499ab927933a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f887 expref: 99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230099:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x730c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.230103:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.230107:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x730b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.230108:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.230111:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.230115:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 4/0,1 mode: --/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230131:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000020ea6998/0xf62499ab927933a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f887 expref: 99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.230141:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c7bc1f10/0xf62499ab92793394 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f879 expref: 99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230144:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000020ea6998/0xf62499ab927933a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f887 expref: 99 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.230150:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c7bc1f10/0xf62499ab92793394 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f879 expref: 99 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.230162:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510762496:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.230162:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0000_UUID:14210:1871938510762368:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.230200:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.230247:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955ff100/2, rc 0, transno 4295070754 00000100:00100000:0.0:1785220582.230256:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510758656:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.230284:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d310000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510762624:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.230307:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.230324:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1785220582.230330:0:14207:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a57b8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72dd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.230337:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa62d80/2, rc 0, transno 4295070753 00000100:00100000:0.0:1785220582.230344:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa62d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510757504:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.230368:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.230374:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510762752:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.230395:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.230419:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+1324:3654:x1871938510758528:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13968us (14732us total) trans 4295070179 rc 0/0 00000100:00100000:1.0:1785220582.230425:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284703 00000100:00100000:1.0:1785220582.230439:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510762624 00000100:00100000:1.0:1785220582.230442:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510762624 00000100:00100000:1.0:1785220582.230463:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510762752 00000100:00100000:1.0:1785220582.230466:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510762752 00000100:00100000:1.0:1785220582.230485:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284713 00000100:00100000:0.0:1785220582.230489:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284714 00000100:00100000:1.0:1785220582.230491:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+132:3653:x1871938510762624:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.230496:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d72fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+132:3655:x1871938510762752:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.230502:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29453 count 1 00002000:00080000:0.0:1785220582.230504:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29454 count 1 00010000:00010000:1.0:1785220582.230512:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230513:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230525:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001aaf47d2/0xf62499ab927933be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8a3 expref: 99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.230528:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b1fb3097/0xf62499ab927933b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f895 expref: 99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230533:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x730e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.230535:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x730d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.230536:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.230538:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.230540:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.230541:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.230559:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b1fb3097/0xf62499ab927933b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f895 expref: 99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230559:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001aaf47d2/0xf62499ab927933be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8a3 expref: 99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.230570:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b1fb3097/0xf62499ab927933b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f895 expref: 99 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.230570:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001aaf47d2/0xf62499ab927933be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8a3 expref: 99 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.230581:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510762880:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.230581:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510763008:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.230598:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.230622:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6e40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.230637:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.230684:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d96667b80/2, rc 0, transno 4295070178 00000100:00100000:0.0:1785220582.230684:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510763136:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.230692:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d96667b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510760192:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.230702:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbf800/2, rc 0, transno 4295070177 00000100:00100000:2.0:1785220582.230706:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510760064:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.230722:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97adf100/2, rc 0, transno 4295070751 00000100:00100000:2.0:1785220582.230732:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97adf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510759296:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.230741:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fe300/2, rc 0, transno 4295070175 00000100:00100000:2.0:1785220582.230743:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510758784:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.230760:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955ff800/2, rc 0, transno 4295070174 00000100:00100000:2.0:1785220582.230763:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510758400:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.230815:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510763264:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.230960:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.230967:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.231001:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.231025:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510763392:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.231048:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.231190:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.231196:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91583100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510763776:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.231219:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.231288:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.231304:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510763392 00000100:00100000:3.0:1785220582.231309:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510763392 00000100:00100000:0.0:1785220582.231415:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510763648:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.231438:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.231459:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.231464:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510763520:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.231481:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.231723:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.231739:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510763776 00000100:00100000:3.0:1785220582.231744:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510763776 00000100:00100000:0.0:1785220582.231781:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988864 00000100:00100000:0.0:1785220582.231786:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988864 00010000:00010000:0.0:1785220582.231807:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e8c4afe7/0xf62499ab9279254e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f642 expref: 128 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.231838:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100639 00000100:00100000:0.0:1785220582.231847:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+128:2406:x1871938519988864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.231858:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.231864:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e8c4afe7/0xf62499ab9279254e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f642 expref: 128 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.231876:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e8c4afe7/0xf62499ab9279254e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f642 expref: 128 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.231895:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8c4afe7/0xf62499ab9279254e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f642 expref: 128 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.231909:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.231913:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.231918:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.231928:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.231953:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.231977:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2406:x1871938519988864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 131us (255us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.231984:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100639 00010000:00010000:0.0:1785220582.232008:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232017:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232026:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232034:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232048:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.232197:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d311180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1325:3655:x1871938510760704:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14109us (14417us total) trans 4295070180 rc 0/0 00000100:00100000:0.0:1785220582.232204:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284706 00000100:00100000:0.0:1785220582.232219:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510763136 00000100:00100000:0.0:1785220582.232222:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510763136 00000100:00100000:0.0:1785220582.232242:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510763648 00000100:00100000:0.0:1785220582.232245:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510763648 00000100:00100000:0.0:1785220582.232260:0:14256:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510763520 00000100:00100000:0.0:1785220582.232263:0:14256:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510763520 00000100:00100000:0.0:1785220582.232280:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284715 00000100:00100000:0.0:1785220582.232286:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1327:3653:x1871938510763392:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.232302:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29414 count 1 00010000:00010000:0.0:1785220582.232317:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232333:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000025acc16/0xf62499ab927927a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f67a expref: 126 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232339:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e6:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.232342:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.232345:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232367:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000025acc16/0xf62499ab927927a1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f67a expref: 126 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232378:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000025acc16/0xf62499ab927927a1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f67a expref: 126 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.232388:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510763904:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.232422:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.232437:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284716 00000100:00100000:0.0:1785220582.232444:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1327:3655:x1871938510763136:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.232457:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29412 count 1 00010000:00010000:0.0:1785220582.232467:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232478:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007d6f06f3/0xf62499ab9279265f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f65e expref: 126 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232488:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e4:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.232491:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.232494:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232512:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007d6f06f3/0xf62499ab9279265f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f65e expref: 126 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232522:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007d6f06f3/0xf62499ab9279265f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f65e expref: 126 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.232532:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510764032:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.232554:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.232573:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284717 00000100:00100000:0.0:1785220582.232582:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1327:3655:x1871938510763648:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.232595:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29416 count 1 00010000:00010000:0.0:1785220582.232606:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232618:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cba269da/0xf62499ab927928a4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f696 expref: 126 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232625:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e8:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.232628:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.232632:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232650:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cba269da/0xf62499ab927928a4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f696 expref: 126 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.232658:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cba269da/0xf62499ab927928a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f696 expref: 126 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.232667:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510764160:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.232687:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.232705:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284718 00000100:00100000:0.0:1785220582.232712:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1327:3655:x1871938510763520:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.232725:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29415 count 1 00010000:00010000:0.0:1785220582.232737:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234264:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000da42429e/0xf62499ab9279280a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f688 expref: 126 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234287:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e7:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.234291:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.234296:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234334:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000da42429e/0xf62499ab9279280a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f688 expref: 126 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234347:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000da42429e/0xf62499ab9279280a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f688 expref: 126 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.234361:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510764288:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.234415:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f300 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.234438:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.234444:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234453:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.234457:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234467:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234483:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234491:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a57d4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234514:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.234516:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.234518:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.234574:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97841c00/2, rc 0, transno 4295070180 00000100:00100000:0.0:1785220582.234591:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510760704:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.234648:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510764416:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.234685:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.234714:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510764416 00000100:00100000:0.0:1785220582.234719:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510764416 00000100:00100000:0.0:1785220582.234767:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284719 00000100:00100000:0.0:1785220582.234776:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1328:3653:x1871938510764416:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.234795:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29417 count 1 00010000:00010000:0.0:1785220582.234812:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234827:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000130bf19a/0xf62499ab92792930 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6a4 expref: 126 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234834:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e9:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.234837:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.234841:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234861:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000130bf19a/0xf62499ab92792930 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6a4 expref: 126 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.234871:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000130bf19a/0xf62499ab92792930 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6a4 expref: 126 pid: 14166 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.234888:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510764544:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.234913:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.235309:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fc380/2, rc 0, transno 4295070179 00000100:00100000:2.0:1785220582.235325:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510758528:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.235349:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.235442:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510763264 00000100:00100000:1.0:1785220582.235447:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510763264 00000100:00100000:1.0:1785220582.235479:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284720 00000100:00100000:1.0:1785220582.235486:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1329:3652:x1871938510763264:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.235500:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29413 count 1 00010000:00010000:1.0:1785220582.235514:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.235531:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eafc984f/0xf62499ab927926e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f66c expref: 126 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.235539:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72e5:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.235542:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.235546:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.235568:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eafc984f/0xf62499ab927926e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f66c expref: 126 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.235578:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eafc984f/0xf62499ab927926e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f66c expref: 126 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.235595:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c680000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510764672:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.235700:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.236078:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.236198:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510764800:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.236231:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.237457:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510764800 00000100:00100000:1.0:1785220582.237466:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510764800 00000100:00100000:1.0:1785220582.237517:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284721 00000100:00100000:1.0:1785220582.237563:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1330:3655:x1871938510764800:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.237577:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284722 00000100:00100000:3.0:1785220582.237586:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+133:3653:x1871938510763776:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.237591:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29418 count 1 00002000:00080000:3.0:1785220582.237597:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29455 count 1 00010000:00010000:3.0:1785220582.237613:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.237621:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.237634:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b76efe31/0xf62499ab927933cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b1 expref: 99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.237634:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000060a251d4/0xf62499ab927929a0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b2 expref: 126 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.237643:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x730f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.237643:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ea:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.237648:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.237648:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.237653:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.237653:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.237682:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b76efe31/0xf62499ab927933cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b1 expref: 99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.237682:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000060a251d4/0xf62499ab927929a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b2 expref: 126 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.237695:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000060a251d4/0xf62499ab927929a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b2 expref: 126 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.237696:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b76efe31/0xf62499ab927933cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b1 expref: 99 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.237709:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510764928:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.237709:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c681880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510765056:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.237775:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.237804:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bae40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.238031:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93761180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0000_UUID:14196:1871938510761344:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.238050:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.238488:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.238531:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510762496:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.238544:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.238573:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.238606:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.238634:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.238653:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519988992 00000100:00100000:2.0:1785220582.238657:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519988992 00010000:00010000:2.0:1785220582.238682:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f6b09c95/0xf62499ab92793378 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f85d expref: 100 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.238705:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519989248 00000100:00100000:2.0:1785220582.238709:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519989248 00010000:00010000:2.0:1785220582.238721:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f7bce6c9/0xf62499ab92793386 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f86b expref: 101 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.238738:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519989504 00000100:00100000:2.0:1785220582.238741:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519989504 00000100:00100000:1.0:1785220582.238742:0:13874:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:13874:1871938510761728:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.238767:0:13874:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.238789:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519989120 00000100:00100000:1.0:1785220582.238792:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519989120 00010000:00010000:1.0:1785220582.238808:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000de3a5785/0xf62499ab9279336a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f84f expref: 103 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.238832:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100640 00000100:00100000:1.0:1785220582.238839:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+103:2406:x1871938519988992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.238852:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.238858:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f6b09c95/0xf62499ab92793378 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f85d expref: 103 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.238869:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f6b09c95/0xf62499ab92793378 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f85d expref: 103 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.238888:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6b09c95/0xf62499ab92793378 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f85d expref: 103 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.238901:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7309:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.238906:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7309:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.238911:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.238926:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000020ea6998/0xf62499ab927933a2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f887 expref: 102 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.238926:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.238946:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.238949:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100641 00000100:00100000:2.0:1785220582.238955:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+102:2406:x1871938519989248:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.238963:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.238967:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f7bce6c9/0xf62499ab92793386 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f86b expref: 102 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.238973:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+102:2406:x1871938519988992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 135us (488us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.238976:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f7bce6c9/0xf62499ab92793386 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f86b expref: 102 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.238980:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100640 00010000:00010000:2.0:1785220582.238989:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f7bce6c9/0xf62499ab92793386 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f86b expref: 102 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.238997:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100642 00010000:00010000:2.0:1785220582.238998:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.239002:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730a:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.239002:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+100:2406:x1871938519989120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.239006:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239009:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.239014:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239014:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000de3a5785/0xf62499ab9279336a lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f84f expref: 100 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239025:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000de3a5785/0xf62499ab9279336a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f84f expref: 100 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239030:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.239036:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7308:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.239039:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7308:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.239043:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239046:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+100:2406:x1871938519989248:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (473us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.239051:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239053:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100641 00010000:00010000:1.0:1785220582.239062:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.239065:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100643 00000100:00100000:2.0:1785220582.239071:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+99:2406:x1871938519989504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.239077:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.239077:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+99:2406:x1871938519989120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (443us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.239082:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000020ea6998/0xf62499ab927933a2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f887 expref: 99 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239112:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000020ea6998/0xf62499ab927933a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f887 expref: 99 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239125:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.239128:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.239132:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239139:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239157:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.239176:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+99:2406:x1871938519989504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (569us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.239182:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100643 00000100:00100000:1.0:1785220582.239189:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100642 00010000:00010000:1.0:1785220582.239205:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239215:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.239215:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239229:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239236:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.239238:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239252:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.239253:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.239265:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519989376 00000100:00100000:2.0:1785220582.239268:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519989376 00010000:00010000:2.0:1785220582.239283:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009c39f102/0xf62499ab927925cc lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f650 expref: 127 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239300:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519989632 00000100:00100000:2.0:1785220582.239302:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519989632 00010000:00010000:2.0:1785220582.239310:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c7bc1f10/0xf62499ab92793394 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f879 expref: 98 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239325:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519989760 00000100:00100000:2.0:1785220582.239328:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519989760 00010000:00010000:2.0:1785220582.239337:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001aaf47d2/0xf62499ab927933be lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8a3 expref: 99 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239354:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100644 00000100:00100000:2.0:1785220582.239359:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2406:x1871938519989376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.239367:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.239371:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009c39f102/0xf62499ab927925cc lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f650 expref: 127 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239380:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009c39f102/0xf62499ab927925cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f650 expref: 127 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239393:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.239396:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.239400:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239408:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.239412:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93760a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+134:3652:x1871938510761216:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17391us (17460us total) trans 4295070755 rc 0/0 00010000:00010000:2.0:1785220582.239419:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.239419:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284707 00000100:00100000:2.0:1785220582.239436:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2406:x1871938519989376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (220us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.239438:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1785220582.239443:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100644 00010000:00010000:1.0:1785220582.239444:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239452:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:2.0:1785220582.239454:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100645 00010000:00010000:1.0:1785220582.239457:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239459:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+99:2406:x1871938519989632:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.239466:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.239467:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239470:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c7bc1f10/0xf62499ab92793394 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f879 expref: 99 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239480:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c7bc1f10/0xf62499ab92793394 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f879 expref: 99 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239480:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239489:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f228b1ca/0xf62499ab927a57db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239498:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7bc1f10/0xf62499ab92793394 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f879 expref: 99 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239505:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.239507:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.239507:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.239509:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.239509:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.239513:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239516:0:13874:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239521:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239524:0:13874:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.239532:0:13874:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239536:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.239539:0:13874:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.239548:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+98:2406:x1871938519989632:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (312us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.239550:0:13874:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.239554:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100645 00000100:00100000:2.0:1785220582.239582:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100646 00000100:00100000:2.0:1785220582.239588:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+97:2406:x1871938519989760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.239595:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.239600:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001aaf47d2/0xf62499ab927933be lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8a3 expref: 97 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239609:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001aaf47d2/0xf62499ab927933be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8a3 expref: 97 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239626:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.239629:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.239633:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239640:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239651:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.239780:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+97:2406:x1871938519989760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 191us (526us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.239787:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100646 00010000:00010000:2.0:1785220582.239808:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239816:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239833:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239840:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.239854:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220582.240000:0:14251:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de3a5785/0xf62499ab9279336a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f84f expref: 96 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.240012:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93761500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510761472:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.240027:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240106:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240114:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240122:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240130:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240145:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220582.240425:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+136:3654:x1871938510761088:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18101us (18159us total) trans 4295070758 rc 0/0 00000100:00100000:3.0:1785220582.240432:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284708 00010000:00010000:3.0:1785220582.240449:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220582.240454:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240462:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.240465:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240473:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240484:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240491:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a57e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240504:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.240507:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240513:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.240516:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240522:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240529:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 1/0,0 mode: --/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240535:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b84298a7/0xf62499ab927a57fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240543:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.240546:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240551:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.240554:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240560:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240572:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240578:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000059cef3a/0xf62499ab927a57e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.240588:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.240590:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.240591:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.240596:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.240675:0:13874:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+135:3653:x1871938510761600:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15993us (16721us total) trans 4295070756 rc 0/0 00000100:00100000:1.0:1785220582.240683:0:13874:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284709 00000100:00100000:2.0:1785220582.240684:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+135:3655:x1871938510762112:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10663us (10904us total) trans 4295070757 rc 0/0 00000100:00100000:2.0:1785220582.240690:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284711 00000100:00100000:1.0:1785220582.240719:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510762880:192.168.201.51@tcp:104: 00000004:00080000:2.0:1785220582.240721:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93760380/2, rc 0, transno 4295070758 00000100:00100000:2.0:1785220582.240730:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510761088:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.240733:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.240763:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93760700/2, rc 0, transno 4295070755 00000100:00100000:2.0:1785220582.240768:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93760700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510761216:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.240783:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fed80/2, rc 0, transno 4295070757 00000100:00100000:1.0:1785220582.240787:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510762112:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.240800:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.240826:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519989888 00000100:00100000:1.0:1785220582.240831:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519989888 00010000:00010000:1.0:1785220582.240850:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b1fb3097/0xf62499ab927933b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f895 expref: 96 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.240885:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100647 00000100:00100000:1.0:1785220582.240893:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+96:2406:x1871938519989888:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.240903:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.240908:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b1fb3097/0xf62499ab927933b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f895 expref: 96 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.240918:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b1fb3097/0xf62499ab927933b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f895 expref: 96 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.240932:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1fb3097/0xf62499ab927933b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f895 expref: 96 pid: 14252 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.240943:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.240946:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.240951:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.240959:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.240976:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.240995:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+95:2406:x1871938519989888:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (194us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.241001:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100647 00010000:00010000:1.0:1785220582.241018:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.241027:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.241027:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.241036:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.241040:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990144 00000100:00100000:2.0:1785220582.241043:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990144 00010000:00010000:1.0:1785220582.241044:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.241056:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.241059:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000025acc16/0xf62499ab927927a1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f67a expref: 127 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.241078:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100648 00000100:00100000:2.0:1785220582.241108:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2407:x1871938519990144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.241118:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.241124:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000025acc16/0xf62499ab927927a1 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f67a expref: 127 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241133:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000025acc16/0xf62499ab927927a1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f67a expref: 127 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241146:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.241149:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.241154:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241162:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241175:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.241189:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2407:x1871938519990144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.241195:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100648 00000100:00100000:2.0:1785220582.241226:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.241239:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990016 00000100:00100000:2.0:1785220582.241242:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990016 00010000:00010000:2.0:1785220582.241255:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000130bf19a/0xf62499ab92792930 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6a4 expref: 127 pid: 14166 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.241272:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100649 00000100:00100000:2.0:1785220582.241278:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2406:x1871938519990016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.241286:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.241291:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000130bf19a/0xf62499ab92792930 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6a4 expref: 127 pid: 14166 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241300:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000130bf19a/0xf62499ab92792930 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6a4 expref: 127 pid: 14166 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241312:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.241316:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.241321:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241329:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.241340:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.241353:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2406:x1871938519990016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (127us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.241359:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100649 00000100:00100000:2.0:1785220582.241389:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.241409:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.241467:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d915aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+134:3653:x1871938510762624:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10975us (11158us total) trans 4295070759 rc 0/0 00000100:00100000:1.0:1785220582.241473:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284713 00010000:00010000:1.0:1785220582.241490:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.241495:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.241503:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.241507:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.241520:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.241533:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.241541:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052d18d1d/0xf62499ab927a5805 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.241556:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.241557:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.241559:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.241568:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990272 00000100:00100000:1.0:1785220582.241571:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990272 00010000:00010000:1.0:1785220582.241586:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007d6f06f3/0xf62499ab9279265f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f65e expref: 127 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.241603:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990400 00000100:00100000:1.0:1785220582.241606:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990400 00000100:00100000:2.0:1785220582.241663:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.241773:0:14185:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c39f102/0xf62499ab927925cc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f650 expref: 128 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.241785:0:14185:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d313480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_042:lustre-OST0001_UUID:14185:1871938510761984:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.241801:0:14185:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.241810:0:14185:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.241815:0:14185:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.241880:0:14185:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.241888:0:14185:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.241910:0:14185:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.241942:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cba269da/0xf62499ab927928a4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f696 expref: 127 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.241960:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990528 00000100:00100000:1.0:1785220582.241964:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990528 00010000:00010000:1.0:1785220582.241975:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000da42429e/0xf62499ab9279280a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f688 expref: 128 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.241998:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100650 00000100:00100000:1.0:1785220582.242004:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+128:2406:x1871938519990272:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.242012:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220582.242012:0:14185:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea6935450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_042:lustre-MDT0000-mdtlov_UUID+1331:3654:x1871938510761856:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15478us (16597us total) trans 4295070181 rc 0/0 00010000:00010000:1.0:1785220582.242017:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007d6f06f3/0xf62499ab9279265f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f65e expref: 128 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.242019:0:14185:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284710 00010000:00010000:1.0:1785220582.242027:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007d6f06f3/0xf62499ab9279265f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f65e expref: 128 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242039:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.242042:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.242045:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242053:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242064:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.242077:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+128:2406:x1871938519990272:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 73us (687us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.242083:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100650 00010000:00010000:0.0:1785220582.242161:0:14210:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020ea6998/0xf62499ab927933a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f887 expref: 94 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.242174:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0000_UUID:14210:1871938510762368:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.242186:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242194:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242201:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242209:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242215:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242230:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.242253:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100651 00000100:00100000:1.0:1785220582.242259:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2407:x1871938519990400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.242269:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.242274:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cba269da/0xf62499ab927928a4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f696 expref: 127 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242283:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cba269da/0xf62499ab927928a4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f696 expref: 127 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242295:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.242299:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.242303:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.242304:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+134:3653:x1871938510762240:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12270us (12345us total) trans 4295070760 rc 0/0 00000100:00100000:0.0:1785220582.242310:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284712 00010000:00010000:1.0:1785220582.242311:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242322:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.242331:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100652 00000100:00100000:1.0:1785220582.242334:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2407:x1871938519990400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (926us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.242338:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c304050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+127:2406:x1871938519990528:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.242340:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100651 00010000:00010000:0.0:1785220582.242345:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.242350:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000da42429e/0xf62499ab9279280a lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f688 expref: 126 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242359:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000da42429e/0xf62499ab9279280a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f688 expref: 126 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242372:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.242375:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.242379:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242387:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242399:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.242408:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c304050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+126:2406:x1871938519990528:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 72us (747us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.242413:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100652 00010000:00010000:0.0:1785220582.242445:0:14168:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001aaf47d2/0xf62499ab927933be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8a3 expref: 93 pid: 14203 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.242452:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510763008:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.242461:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242466:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242470:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242475:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242479:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242491:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220582.242513:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.242518:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242525:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.242529:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242541:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242553:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242560:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3a898ca/0xf62499ab927a57f0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.242562:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d72fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+134:3655:x1871938510762752:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12066us (12166us total) trans 4295070761 rc 0/0 00000100:00100000:0.0:1785220582.242568:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284714 00010000:00010000:1.0:1785220582.242577:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.242580:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242586:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.242586:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.242589:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242590:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242597:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242598:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.242601:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242609:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242620:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242627:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094b7a2d3/0xf62499ab927a580c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242641:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.242643:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.242644:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.242666:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000130bf19a/0xf62499ab92792930 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6a4 expref: 125 pid: 14166 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242669:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.242676:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510764544:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.242678:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000722c2c8e/0xf62499ab927a57f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242688:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242695:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242696:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.242697:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.242699:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.242703:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242710:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.242721:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72d880/2, rc 0, transno 4295070761 00010000:00010000:0.0:1785220582.242722:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.242726:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510762752:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.242734:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.242808:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d95a0c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+1331:3653:x1871938510764416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8034us (8123us total) trans 4295070182 rc 0/0 00000100:00100000:0.0:1785220582.242813:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284719 00010000:00010000:0.0:1785220582.242890:0:14191:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007d6f06f3/0xf62499ab9279265f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f65e expref: 124 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.242897:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510764032:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.242906:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242911:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242915:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242920:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242924:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.242936:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.242958:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.242963:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242970:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922aca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.242974:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242981:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.242992:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243000:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000812fbe23/0xf62499ab927a582f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.243007:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1331:3655:x1871938510763136:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10563us (11719us total) trans 4295070183 rc 0/0 00000100:00100000:0.0:1785220582.243013:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284716 00010000:00010000:1.0:1785220582.243014:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.243017:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243028:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.243031:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243038:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243040:0:14205:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000da42429e/0xf62499ab9279280a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f688 expref: 123 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243048:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.243051:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510764288:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.243056:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001fa329aa/0xf62499ab927a581a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243061:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243068:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.243068:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243069:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.243071:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.243076:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243101:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243110:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243124:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.243192:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b48000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1331:3655:x1871938510763520:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10481us (11709us total) trans 4295070184 rc 0/0 00000100:00100000:0.0:1785220582.243199:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284718 00010000:00010000:0.0:1785220582.243215:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.243218:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243225:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.243228:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243235:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243246:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243252:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a7f992b7/0xf62499ab927a5828 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243266:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.243268:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.243270:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.243276:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.243277:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921f9180/2, rc 0, transno 4295070183 00000100:00100000:1.0:1785220582.243288:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510763136:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.243294:0:14256:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000025acc16/0xf62499ab927927a1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f67a expref: 122 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.243300:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921f8000/2, rc 0, transno 4295070184 00000100:00100000:1.0:1785220582.243304:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510763520:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.243310:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0001_UUID:14256:1871938510763904:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.243321:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243330:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243337:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243345:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243352:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243368:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.243385:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.243390:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243397:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.243400:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243408:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243418:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243425:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d9f8f45c/0xf62499ab927a5813 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.243432:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+1331:3653:x1871938510763392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11147us (12384us total) trans 4295070185 rc 0/0 00010000:00010000:1.0:1785220582.243439:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.243439:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284715 00010000:00010000:1.0:1785220582.243440:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.243442:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.243456:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.243508:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510765184:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.243542:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.243562:0:14245:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510765184 00000100:00100000:0.0:1785220582.243566:0:14245:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510765184 00000100:00100000:1.0:1785220582.243577:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510765696:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.243592:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284723 00000100:00100000:1.0:1785220582.243599:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.243599:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1331:3655:x1871938510765184:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.243620:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29419 count 1 00010000:00010000:0.0:1785220582.243638:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243652:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4c9eb4f/0xf62499ab92792a4f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c0 expref: 121 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.243660:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72eb:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.243664:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.243669:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.243685:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:0.0:1785220582.243690:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4c9eb4f/0xf62499ab92792a4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c0 expref: 121 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.243691:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b10ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510765568:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.243702:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4c9eb4f/0xf62499ab92792a4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c0 expref: 121 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.243710:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.243713:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510765952:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.243745:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.243769:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510765696 00000100:00100000:0.0:1785220582.243773:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510765696 00000100:00100000:0.0:1785220582.243794:0:14195:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510765568 00000100:00100000:0.0:1785220582.243796:0:14195:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510765568 00000100:00100000:1.0:1785220582.243805:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284724 00000100:00100000:1.0:1785220582.243812:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+135:3655:x1871938510765696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.243825:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29458 count 1 00010000:00010000:1.0:1785220582.243838:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243851:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000020d4d99e/0xf62499ab927933f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8db expref: 92 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.243858:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7312:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.243861:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7312:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.243866:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.243869:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.243885:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000020d4d99e/0xf62499ab927933f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8db expref: 92 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.243895:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93762680/2, rc 0, transno 4295070181 00010000:00010000:1.0:1785220582.243895:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000020d4d99e/0xf62499ab927933f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8db expref: 92 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.243904:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93762680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510761856:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.243905:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510766080:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.243924:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.244013:0:14195:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284725 00000100:00100000:0.0:1785220582.244022:0:14195:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+135:3655:x1871938510765568:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.244035:0:14195:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29457 count 1 00010000:00010000:0.0:1785220582.244046:0:14195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.244058:0:14195:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000099d2b6a3/0xf62499ab927933e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8cd expref: 92 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.244065:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7311:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.244068:0:14195:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7311:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.244072:0:14195:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.244176:0:14195:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000099d2b6a3/0xf62499ab927933e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8cd expref: 92 pid: 14173 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.244186:0:14195:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000099d2b6a3/0xf62499ab927933e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8cd expref: 92 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.244195:0:14195:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510766336:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.244218:0:14195:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.244245:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510765440:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.244271:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.244291:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510766208:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.244303:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510765440 00000100:00100000:1.0:1785220582.244307:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510765440 00000100:00100000:0.0:1785220582.244312:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.244389:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510766208 00000100:00100000:1.0:1785220582.244392:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510766208 00000004:00080000:0.0:1785220582.244399:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0c700/2, rc 0, transno 4295070182 00000100:00100000:0.0:1785220582.244408:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510764416:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.244412:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284726 00000100:00100000:1.0:1785220582.244418:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+136:3653:x1871938510765440:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.244429:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29456 count 1 00010000:00010000:1.0:1785220582.244440:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.244446:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921fbb80/2, rc 0, transno 4295070185 00000100:00100000:0.0:1785220582.244452:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510763392:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.244458:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000030a01391/0xf62499ab927933da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8bf expref: 92 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.244465:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7310:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.244468:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7310:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.244472:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.244477:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d310000/2, rc 0, transno 4295070759 00000100:00100000:0.0:1785220582.244484:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d310000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510762624:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.244489:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000030a01391/0xf62499ab927933da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8bf expref: 92 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.244498:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000030a01391/0xf62499ab927933da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8bf expref: 92 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.244507:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510766720:192.168.201.51@tcp:104: 00000004:00080000:0.0:1785220582.244507:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d312d80/2, rc 0, transno 4295070760 00000100:00100000:0.0:1785220582.244513:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510762240:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.244526:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9780 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.244540:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d311c00/2, rc 0, transno 4295070756 00000100:00100000:0.0:1785220582.244546:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510761600:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.244570:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510766976:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.244591:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.244614:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1785220582.244620:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510765312:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.244641:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.244656:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510765824:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.244673:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.244689:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510766976 00000100:00100000:0.0:1785220582.244689:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d310700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510766848:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.244693:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510766976 00000100:00100000:0.0:1785220582.244707:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.244713:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510765312 00000100:00100000:1.0:1785220582.244716:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510765312 00000100:00100000:1.0:1785220582.244736:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510765824 00000100:00100000:1.0:1785220582.244738:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510765824 00000100:00100000:0.0:1785220582.244739:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510766848 00000100:00100000:0.0:1785220582.244741:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510766848 00000100:00100000:1.0:1785220582.244769:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284727 00000100:00100000:1.0:1785220582.244775:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1333:3653:x1871938510766208:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.244787:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29421 count 1 00010000:00010000:1.0:1785220582.244800:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.244812:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd232126/0xf62499ab92792b44 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6dc expref: 121 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.244819:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ed:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.244822:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.244826:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.244845:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd232126/0xf62499ab92792b44 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6dc expref: 121 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.244855:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd232126/0xf62499ab92792b44 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6dc expref: 121 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.244865:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0001_UUID:14232:1871938510767232:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.244883:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.244927:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cba269da/0xf62499ab927928a4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f696 expref: 121 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.244939:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510764160:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.244956:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.244965:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.244971:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.244979:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.244987:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.245004:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.245120:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98b49f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1334:3655:x1871938510763648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12539us (13681us total) trans 4295070186 rc 0/0 00000100:00100000:0.0:1785220582.245224:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284728 00000100:00100000:0.0:1785220582.245233:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+139:3653:x1871938510766976:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.245244:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29461 count 1 00010000:00010000:0.0:1785220582.245257:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245269:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000aaaab10a/0xf62499ab92793420 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f905 expref: 92 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245276:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7315:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.245279:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7315:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.245282:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245300:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000aaaab10a/0xf62499ab92793420 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f905 expref: 92 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245309:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000aaaab10a/0xf62499ab92793420 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f905 expref: 92 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.245319:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510767360:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.245344:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.245361:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510766592:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.245381:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.245403:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510766592 00000100:00100000:0.0:1785220582.245407:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510766592 00000100:00100000:0.0:1785220582.245430:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284729 00000100:00100000:0.0:1785220582.245437:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1335:3653:x1871938510765312:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.245449:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29420 count 1 00010000:00010000:0.0:1785220582.245460:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245472:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d1a469f3/0xf62499ab92792adb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ce expref: 120 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245479:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ec:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.245481:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.245485:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.245486:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284730 00000100:00100000:1.0:1785220582.245495:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+139:3653:x1871938510765824:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.245502:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d1a469f3/0xf62499ab92792adb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ce expref: 120 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.245505:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29459 count 1 00010000:00010000:0.0:1785220582.245511:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d1a469f3/0xf62499ab92792adb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ce expref: 120 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.245517:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.245520:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510767488:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.245528:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000018f902d4/0xf62499ab92793404 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e9 expref: 92 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.245535:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7313:0x0].0x0 since=17s v=(0 0) 00000100:00100000:0.0:1785220582.245536:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.245538:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7313:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.245542:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.245551:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284731 00000100:00100000:0.0:1785220582.245557:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+139:3653:x1871938510766848:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.245559:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000018f902d4/0xf62499ab92793404 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e9 expref: 92 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.245567:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29460 count 1 00010000:00010000:1.0:1785220582.245568:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000018f902d4/0xf62499ab92793404 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e9 expref: 92 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.245577:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510767616:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.245577:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245587:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003071c390/0xf62499ab92793412 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f7 expref: 92 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.245595:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.245599:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7314:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.245602:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7314:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.245606:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245623:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003071c390/0xf62499ab92793412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f7 expref: 92 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245632:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003071c390/0xf62499ab92793412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f7 expref: 92 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.245641:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510767744:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.245658:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.245681:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921fa680/2, rc 0, transno 4295070186 00000100:00100000:0.0:1785220582.245689:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510763648:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.245696:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284732 00000100:00100000:1.0:1785220582.245703:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1335:3655:x1871938510766592:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.245713:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29423 count 1 00000100:00100000:0.0:1785220582.245724:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510767872:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.245725:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.245736:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fba1fe8e/0xf62499ab92792ca2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f8 expref: 120 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.245743:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ef:0x0].0x0 since=18s v=(0 0) 00000100:00100000:0.0:1785220582.245745:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.245747:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.245762:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.245777:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510767872 00000100:00100000:0.0:1785220582.245780:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510767872 00010000:00010000:1.0:1785220582.245782:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fba1fe8e/0xf62499ab92792ca2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f8 expref: 120 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.245791:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fba1fe8e/0xf62499ab92792ca2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f8 expref: 120 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.245800:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510768000:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.245802:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284733 00000100:00100000:0.0:1785220582.245807:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1336:3655:x1871938510767872:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.245818:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8df900 going to sleep for 11 seconds 00002000:00080000:0.0:1785220582.245818:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29424 count 1 00010000:00010000:0.0:1785220582.245842:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245853:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a52aaa74/0xf62499ab92792d6d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f706 expref: 120 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245859:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f0:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.245863:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.245866:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245884:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a52aaa74/0xf62499ab92792d6d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f706 expref: 120 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.245894:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a52aaa74/0xf62499ab92792d6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f706 expref: 120 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.245904:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510768128:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.245921:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.245936:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1785220582.245941:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510766464:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.245963:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.245980:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d313480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510767104:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.245998:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.246257:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510766464 00000100:00100000:0.0:1785220582.246262:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510766464 00000100:00100000:0.0:1785220582.246287:0:14145:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510767104 00000100:00100000:0.0:1785220582.246290:0:14145:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510767104 00000100:00100000:0.0:1785220582.246309:0:14145:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284734 00000100:00100000:0.0:1785220582.246315:0:14145:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1337:3655:x1871938510766464:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.246326:0:14145:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29422 count 1 00010000:00010000:0.0:1785220582.246338:0:14145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246349:0:14145:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f5d86f7b/0xf62499ab92792c0f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ea expref: 120 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246355:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ee:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.246358:0:14145:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.246361:0:14145:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246378:0:14145:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f5d86f7b/0xf62499ab92792c0f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ea expref: 120 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246387:0:14145:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f5d86f7b/0xf62499ab92792c0f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ea expref: 120 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.246396:0:14145:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510768256:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.246423:0:14145:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.246438:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284735 00000100:00100000:0.0:1785220582.246445:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+140:3655:x1871938510767104:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.246456:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29462 count 1 00010000:00010000:0.0:1785220582.246469:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246481:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000165cf95d/0xf62499ab9279342e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f913 expref: 92 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246487:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7316:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.246490:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7316:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.246493:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246509:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000165cf95d/0xf62499ab9279342e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f913 expref: 92 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.246517:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000165cf95d/0xf62499ab9279342e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f913 expref: 92 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.246527:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510768384:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.246547:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.246941:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284717 00010000:00010000:3.0:1785220582.246978:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.246985:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.246994:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ade00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.246998:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.247007:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.247026:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.247034:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e45a31b4/0xf62499ab927a5821 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.247052:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.247055:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.247057:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.251615:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c680000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0001_UUID:14176:1871938510764672:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.251618:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.251646:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990656 00010000:00010000:1.0:1785220582.251647:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.251652:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990656 00010000:00010000:3.0:1785220582.251689:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eafc984f/0xf62499ab927926e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f66c expref: 121 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.251730:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100653 00000100:00100000:3.0:1785220582.251741:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+121:2407:x1871938519990656:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.251766:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.251772:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eafc984f/0xf62499ab927926e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f66c expref: 121 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.251784:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eafc984f/0xf62499ab927926e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f66c expref: 121 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.251806:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eafc984f/0xf62499ab927926e4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f66c expref: 121 pid: 13837 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.251833:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.251839:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72e5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.251844:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.251854:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.251887:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.251906:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.251914:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.251923:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.251931:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+120:2407:x1871938519990656:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 193us (315us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.251931:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.251939:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100653 00010000:00010000:1.0:1785220582.251948:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.251977:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.252076:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.252134:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+1337:3652:x1871938510763264:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16648us (16783us total) trans 4295070187 rc 0/0 00000100:00100000:1.0:1785220582.252143:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284720 00010000:00010000:1.0:1785220582.252163:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1785220582.252168:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.252169:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.252178:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:3.0:1785220582.252181:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.252182:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.252192:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.252195:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.252206:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.252213:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5836 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.252234:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.252244:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.252246:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.252247:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.252250:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.252267:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510766080:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.252282:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252306:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510766720:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.252320:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252340:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0001_UUID:14232:1871938510767232:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.252351:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252369:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990912 00000100:00100000:1.0:1785220582.252372:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990912 00010000:00010000:1.0:1785220582.252388:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b76efe31/0xf62499ab927933cc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b1 expref: 93 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252408:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519990784 00000100:00100000:1.0:1785220582.252411:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519990784 00010000:00010000:1.0:1785220582.252424:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000060a251d4/0xf62499ab927929a0 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b2 expref: 120 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252440:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991040 00000100:00100000:1.0:1785220582.252443:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991040 00010000:00010000:1.0:1785220582.252453:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4c9eb4f/0xf62499ab92792a4f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c0 expref: 121 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252477:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991296 00000100:00100000:1.0:1785220582.252481:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991296 00010000:00010000:1.0:1785220582.252492:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000099d2b6a3/0xf62499ab927933e8 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8cd expref: 94 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252508:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991168 00000100:00100000:1.0:1785220582.252511:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991168 00010000:00010000:1.0:1785220582.252521:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000020d4d99e/0xf62499ab927933f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8db expref: 95 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252539:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991424 00000100:00100000:1.0:1785220582.252542:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991424 00010000:00010000:1.0:1785220582.252553:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000030a01391/0xf62499ab927933da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8bf expref: 96 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252575:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991552 00000100:00100000:1.0:1785220582.252578:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991552 00010000:00010000:1.0:1785220582.252589:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd232126/0xf62499ab92792b44 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6dc expref: 122 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252680:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100654 00000100:00100000:1.0:1785220582.252690:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97843b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+96:2407:x1871938519990912:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.252701:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.252705:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b76efe31/0xf62499ab927933cc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b1 expref: 96 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.252721:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b76efe31/0xf62499ab927933cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b1 expref: 96 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.252737:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.252741:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x730f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.252745:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.252764:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.252768:0:14192:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b76efe31/0xf62499ab927933cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b1 expref: 96 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.252777:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.252779:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510764928:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.252792:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.252803:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97843b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+95:2407:x1871938519990912:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 116us (825us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.252805:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.252812:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.252880:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c681880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510765056:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.252893:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.252935:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.252944:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.252958:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.253076:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+141:3653:x1871938510763776:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15492us (21855us total) trans 4295070762 rc 0/0 00000100:00100000:0.0:1785220582.253099:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284722 00000100:00100000:0.0:1785220582.253141:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510767744:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253158:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.253186:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510768384:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253197:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.253227:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510767488:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253238:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.253270:0:14145:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:14145:1871938510768256:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253286:0:14145:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.253316:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510768128:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253329:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.253358:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510767360:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253371:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.253404:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0001_UUID:14245:1871938510765952:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253417:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.253449:0:14195:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_052:lustre-OST0000_UUID:14195:1871938510766336:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.253468:0:14195:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.253487:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.253497:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.253507:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.253514:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.253525:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.253544:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.253554:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a1c24de/0xf62499ab927a583d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.253576:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.253580:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.253582:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.253628:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91583100/2, rc 0, transno 4295070762 00000100:00100000:0.0:1785220582.253640:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91583100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510763776:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.253704:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.253721:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510768512:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.253776:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.253834:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921fb480/2, rc 0, transno 4295070187 00000100:00100000:0.0:1785220582.253849:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510763264:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.253901:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510768640:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.253936:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.254994:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100655 00000100:00100000:3.0:1785220582.255015:0:14204:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510768512 00000100:00100000:3.0:1785220582.255022:0:14204:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510768512 00000100:00100000:3.0:1785220582.255251:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284736 00000100:00100000:3.0:1785220582.255261:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+141:3653:x1871938510768512:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.255289:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29463 count 1 00010000:00010000:3.0:1785220582.255311:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255329:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001603306c/0xf62499ab9279343c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f921 expref: 95 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255339:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7317:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.255343:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7317:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.255348:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255376:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001603306c/0xf62499ab9279343c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f921 expref: 95 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255388:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001603306c/0xf62499ab9279343c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f921 expref: 95 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.255401:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93763b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510768768:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.255457:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.255475:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510768640 00000100:00100000:3.0:1785220582.255479:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510768640 00000100:00100000:3.0:1785220582.255503:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284737 00000100:00100000:3.0:1785220582.255508:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1337:3653:x1871938510768640:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.255519:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29425 count 1 00010000:00010000:3.0:1785220582.255531:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255542:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000805a98e9/0xf62499ab92792e1c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f714 expref: 122 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255549:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f1:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.255552:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.255556:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255578:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000805a98e9/0xf62499ab92792e1c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f714 expref: 122 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255587:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000805a98e9/0xf62499ab92792e1c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f714 expref: 122 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.255598:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93761500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510768896:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.255619:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.255635:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+122:2407:x1871938519990784:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.255644:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.255651:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000060a251d4/0xf62499ab927929a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b2 expref: 122 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255663:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000060a251d4/0xf62499ab927929a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b2 expref: 122 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255681:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000060a251d4/0xf62499ab927929a0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6b2 expref: 122 pid: 14245 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255694:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.255697:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ea:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.255701:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255714:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255732:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.255760:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+121:2407:x1871938519990784:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 747us (3673us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.255768:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100655 00000100:00100000:3.0:1785220582.255787:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100656 00000100:00100000:3.0:1785220582.255793:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+120:2406:x1871938519991040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.255800:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.255805:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4c9eb4f/0xf62499ab92792a4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c0 expref: 120 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255814:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4c9eb4f/0xf62499ab92792a4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c0 expref: 120 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255886:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4c9eb4f/0xf62499ab92792a4f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6c0 expref: 120 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255897:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.255900:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72eb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.255905:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255918:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255935:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.255946:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255954:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255963:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255971:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.255984:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.256208:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c683100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+1338:3655:x1871938510765184:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12609us (12664us total) trans 4295070188 rc 0/0 00000100:00100000:3.0:1785220582.256215:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284723 00000100:00100000:3.0:1785220582.256242:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+119:2406:x1871938519991040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 449us (4074us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.256248:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100656 00000100:00100000:3.0:1785220582.256259:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100657 00000100:00100000:3.0:1785220582.256265:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+95:2406:x1871938519991296:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.256274:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.256279:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000099d2b6a3/0xf62499ab927933e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8cd expref: 95 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256289:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000099d2b6a3/0xf62499ab927933e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8cd expref: 95 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256302:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099d2b6a3/0xf62499ab927933e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8cd expref: 95 pid: 14173 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256312:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7311:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.256315:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7311:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.256319:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256326:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256343:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.256352:0:14195:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256361:0:14195:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256368:0:14195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256377:0:14195:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256388:0:14195:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.256467:0:14195:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c680a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_052:lustre-MDT0000-mdtlov_UUID+142:3655:x1871938510765568:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12446us (12755us total) trans 4295070763 rc 0/0 00000100:00100000:3.0:1785220582.256473:0:14195:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284725 00010000:00010000:3.0:1785220582.256487:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.256493:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256501:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922adc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.256505:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256514:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256524:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256531:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe56d351/0xf62499ab927a584b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72eb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256547:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.256551:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256557:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.256560:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256567:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256575:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256581:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bd049eb/0xf62499ab927a5859 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256592:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.256593:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.256595:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.256609:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+94:2406:x1871938519991296:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 344us (4426us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.256615:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100657 00000100:00100000:3.0:1785220582.256627:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100658 00000100:00100000:3.0:1785220582.256634:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+93:2407:x1871938519991168:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.256641:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.256646:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000020d4d99e/0xf62499ab927933f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8db expref: 93 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256656:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000020d4d99e/0xf62499ab927933f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8db expref: 93 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256667:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020d4d99e/0xf62499ab927933f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8db expref: 93 pid: 14221 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256676:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7312:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.256679:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7312:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.256683:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256691:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256706:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.256716:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256724:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256732:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256738:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.256763:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.257041:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.257116:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.257154:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.257176:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.257195:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.257279:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.257292:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.257307:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:0.0:1785220582.257345:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b10ad80/2, rc 0, transno 4295070763 00000100:00100000:0.0:1785220582.257356:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b10ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510765568:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.257411:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d72ca80/2, rc 0, transno 4295070764 00000100:00100000:0.0:1785220582.257419:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510765696:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.257449:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0e680/2, rc 0, transno 4295070188 00000100:00100000:0.0:1785220582.257458:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510765184:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.257494:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220582.257504:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100654 00000100:00100000:0.0:1785220582.257506:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510769280:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.257523:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991808 00000100:00100000:1.0:1785220582.257528:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991808 00000100:00100000:0.0:1785220582.257535:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.257549:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000018f902d4/0xf62499ab92793404 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e9 expref: 92 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257557:0:14260:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510769280 00000100:00100000:0.0:1785220582.257560:0:14260:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510769280 00000100:00100000:1.0:1785220582.257565:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992064 00000100:00100000:1.0:1785220582.257567:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992064 00010000:00010000:1.0:1785220582.257577:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fba1fe8e/0xf62499ab92792ca2 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f8 expref: 119 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257584:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284738 00000100:00100000:1.0:1785220582.257588:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992320 00000100:00100000:1.0:1785220582.257591:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992320 00000100:00100000:0.0:1785220582.257591:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98ade680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1338:3653:x1871938510769280:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.257599:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000aaaab10a/0xf62499ab92793420 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f905 expref: 93 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.257608:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29426 count 1 00010000:00010000:0.0:1785220582.257626:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.257640:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005edbefcf/0xf62499ab92792ecb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f722 expref: 119 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.257648:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f2:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.257652:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.257656:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.257680:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005edbefcf/0xf62499ab92792ecb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f722 expref: 119 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.257691:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005edbefcf/0xf62499ab92792ecb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f722 expref: 119 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257703:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98addc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510769408:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.257746:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.257759:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992576 00000100:00100000:1.0:1785220582.257763:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992576 00010000:00010000:1.0:1785220582.257777:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000165cf95d/0xf62499ab9279342e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f913 expref: 94 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.257791:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991680 00000100:00100000:1.0:1785220582.257793:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991680 00010000:00010000:1.0:1785220582.257805:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d1a469f3/0xf62499ab92792adb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ce expref: 120 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257805:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510769152:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.257817:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519991936 00000100:00100000:1.0:1785220582.257819:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519991936 00010000:00010000:1.0:1785220582.257829:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003071c390/0xf62499ab92793412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f7 expref: 95 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257837:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.257840:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992192 00000100:00100000:1.0:1785220582.257843:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992192 00010000:00010000:1.0:1785220582.257854:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a52aaa74/0xf62499ab92792d6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f706 expref: 121 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257856:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510769152 00000100:00100000:0.0:1785220582.257859:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510769152 00000100:00100000:1.0:1785220582.257866:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992448 00000100:00100000:1.0:1785220582.257867:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992448 00010000:00010000:1.0:1785220582.257877:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f5d86f7b/0xf62499ab92792c0f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ea expref: 122 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257881:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284739 00000100:00100000:0.0:1785220582.257887:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d935a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+143:3655:x1871938510769152:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.257897:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100659 00002000:00080000:0.0:1785220582.257898:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29465 count 1 00000100:00100000:1.0:1785220582.257903:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9af450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+95:2407:x1871938519991424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.257910:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.257911:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.257916:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000030a01391/0xf62499ab927933da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8bf expref: 95 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.257922:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003c9b1979/0xf62499ab92793458 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f93d expref: 95 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.257925:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000030a01391/0xf62499ab927933da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8bf expref: 95 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.257928:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7319:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.257932:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7319:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.257935:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.257941:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030a01391/0xf62499ab927933da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8bf expref: 95 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.257950:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7310:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.257953:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7310:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.257953:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003c9b1979/0xf62499ab92793458 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f93d expref: 94 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.257958:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.257962:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003c9b1979/0xf62499ab92793458 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f93d expref: 94 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.257967:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.257973:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0000_UUID:14202:1871938510769536:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.257984:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.257997:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.258003:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9af450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+94:2407:x1871938519991424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 100us (5769us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.258010:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100659 00000100:00100000:1.0:1785220582.258026:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100660 00000100:00100000:0.0:1785220582.258027:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.258032:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+122:2407:x1871938519991552:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.258033:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510769024:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.258041:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.258046:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd232126/0xf62499ab92792b44 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6dc expref: 122 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.258054:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.258056:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd232126/0xf62499ab92792b44 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6dc expref: 122 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258068:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd232126/0xf62499ab92792b44 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6dc expref: 122 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258080:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258082:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ed:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258229:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258237:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258263:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.258286:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+121:2407:x1871938519991552:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 253us (6034us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.258292:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100660 00000100:00100000:1.0:1785220582.258308:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100661 00000100:00100000:1.0:1785220582.258314:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+93:2406:x1871938519991808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.258322:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.258327:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000018f902d4/0xf62499ab92793404 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e9 expref: 93 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258336:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000018f902d4/0xf62499ab92793404 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e9 expref: 93 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258348:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7313:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258352:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7313:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258355:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258362:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258374:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.258390:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+93:2406:x1871938519991808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (1350us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.258397:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100661 00000100:00100000:1.0:1785220582.258410:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100662 00000100:00100000:1.0:1785220582.258417:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+120:2406:x1871938519992064:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.258421:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258424:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.258429:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fba1fe8e/0xf62499ab92792ca2 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f8 expref: 120 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258436:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258445:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fba1fe8e/0xf62499ab92792ca2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f8 expref: 120 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258446:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258454:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258458:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258461:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ef:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258465:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258466:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.258472:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258482:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.258500:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+120:2406:x1871938519992064:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (1384us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.258505:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100662 00000100:00100000:1.0:1785220582.258518:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100663 00000100:00100000:1.0:1785220582.258524:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+92:2406:x1871938519992320:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.258531:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.258535:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000aaaab10a/0xf62499ab92793420 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f905 expref: 92 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258544:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000aaaab10a/0xf62499ab92793420 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f905 expref: 92 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258560:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aaaab10a/0xf62499ab92793420 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f905 expref: 92 pid: 14169 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.258571:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7315:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258574:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7315:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.258578:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.258582:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94dbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+1339:3653:x1871938510766208:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13807us (14269us total) trans 4295070189 rc 0/0 00010000:00010000:1.0:1785220582.258586:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.258590:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284727 00010000:00010000:1.0:1785220582.258602:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.258603:0:14232:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510769024 00000100:00100000:0.0:1785220582.258606:0:14232:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510769024 00000100:00100000:0.0:1785220582.258627:0:14232:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284740 00000100:00100000:0.0:1785220582.258633:0:14232:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98ade300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+144:3655:x1871938510769024:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.258645:0:14232:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29464 count 1 00010000:00010000:0.0:1785220582.258657:0:14232:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258669:0:14232:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009c7a193c/0xf62499ab9279344a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f92f expref: 91 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258676:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7318:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.258680:0:14232:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7318:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.258683:0:14232:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258704:0:14232:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009c7a193c/0xf62499ab9279344a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f92f expref: 91 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258713:0:14232:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009c7a193c/0xf62499ab9279344a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f92f expref: 91 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.258724:0:14232:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510769664:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.258765:0:14232:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.258779:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258786:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258793:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258799:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258815:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.258910:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+145:3653:x1871938510766976:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13679us (14317us total) trans 4295070765 rc 0/0 00000100:00100000:0.0:1785220582.258916:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284728 00010000:00010000:0.0:1785220582.258936:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220582.258942:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258949:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.258952:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258960:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258972:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258978:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5852 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.258996:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.258999:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259004:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.259007:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259018:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259028:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259035:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d95ab63b/0xf62499ab927a5867 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ed:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259061:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.259065:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259072:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ace00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.259075:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259082:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259115:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259237:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000460532ec/0xf62499ab927a586e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259258:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.259260:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.259262:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.259290:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d311180/2, rc 0, transno 4295070765 00000100:00100000:0.0:1785220582.259300:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510766976:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.259346:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921fb800/2, rc 0, transno 4295070189 00000100:00100000:0.0:1785220582.259354:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510766208:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.259391:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98adca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510769792:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.259418:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.259440:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510769792 00000100:00100000:0.0:1785220582.259444:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510769792 00000100:00100000:0.0:1785220582.259469:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284741 00000100:00100000:0.0:1785220582.259476:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+145:3653:x1871938510769792:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.259490:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29466 count 1 00010000:00010000:0.0:1785220582.259503:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259516:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000720211d4/0xf62499ab92793466 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f94b expref: 91 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259522:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x731a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.259525:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.259529:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259553:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000720211d4/0xf62499ab92793466 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f94b expref: 91 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.259564:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000720211d4/0xf62499ab92793466 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f94b expref: 91 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.259573:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0000_UUID:14266:1871938510770048:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.259598:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.260185:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.260196:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510769920:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.260225:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.260331:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+91:2406:x1871938519992320:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1805us (3175us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.260340:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100663 00000100:00100000:1.0:1785220582.260363:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100664 00000100:00100000:1.0:1785220582.260371:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+90:2406:x1871938519992576:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.260381:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.260388:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000165cf95d/0xf62499ab9279342e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f913 expref: 90 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.260400:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000165cf95d/0xf62499ab9279342e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f913 expref: 90 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.260412:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100665 00010000:00010000:1.0:1785220582.260421:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000165cf95d/0xf62499ab9279342e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f913 expref: 90 pid: 14170 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.260427:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510769920 00000100:00100000:2.0:1785220582.260430:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510769920 00010000:00010000:1.0:1785220582.260432:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7316:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.260435:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7316:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.260440:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.260448:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.260465:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.260489:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+89:2406:x1871938519992576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 119us (3312us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.260495:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100664 00000100:00100000:2.0:1785220582.260496:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284742 00000100:00100000:2.0:1785220582.260502:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1339:3655:x1871938510769920:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.260509:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100666 00000100:00100000:1.0:1785220582.260514:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90700850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+88:2406:x1871938519991936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00002000:00080000:2.0:1785220582.260515:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29427 count 1 00010000:00010000:1.0:1785220582.260521:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.260526:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003071c390/0xf62499ab92793412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f7 expref: 88 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.260531:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.260536:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003071c390/0xf62499ab92793412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f7 expref: 88 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.260542:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003dd71c0d/0xf62499ab92792fc7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f730 expref: 119 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.260549:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f3:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.260553:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.260554:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003071c390/0xf62499ab92793412 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f7 expref: 88 pid: 14144 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.260557:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.260563:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7314:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.260566:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7314:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.260570:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.260578:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003dd71c0d/0xf62499ab92792fc7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f730 expref: 119 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.260578:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.260589:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003dd71c0d/0xf62499ab92792fc7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f730 expref: 119 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.260597:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.260600:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510770176:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.261009:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.261019:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.261028:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.261035:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.261049:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.261741:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c681f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+146:3655:x1871938510765696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17926us (18137us total) trans 4295070764 rc 0/0 00000100:00100000:1.0:1785220582.261741:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90700850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+87:2406:x1871938519991936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1222us (4457us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.261759:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284724 00000100:00100000:1.0:1785220582.261759:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100666 00000100:00100000:1.0:1785220582.261783:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100667 00010000:00010000:3.0:1785220582.261786:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220582.261792:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+119:2406:x1871938519992192:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.261793:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.261801:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4dc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.261802:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.261805:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.261808:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a52aaa74/0xf62499ab92792d6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f706 expref: 119 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.261814:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.261819:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a52aaa74/0xf62499ab92792d6d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f706 expref: 119 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.261840:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a52aaa74/0xf62499ab92792d6d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f706 expref: 119 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.261854:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.261858:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.261863:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.261873:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.261896:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.261917:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+118:2406:x1871938519992192:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 127us (4623us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.261924:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100667 00010000:00010000:3.0:1785220582.261926:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.261933:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a589f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.261937:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100668 00000100:00100000:1.0:1785220582.261943:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90cbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+117:2406:x1871938519992448:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.261951:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.261952:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.261954:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.261956:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.261956:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f5d86f7b/0xf62499ab92792c0f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ea expref: 117 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.261964:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.261965:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f5d86f7b/0xf62499ab92792c0f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ea expref: 117 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.261982:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f5d86f7b/0xf62499ab92792c0f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ea expref: 117 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.261988:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+86:2407:x1871938519991168:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 5355us (9791us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.261992:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ee:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.261995:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100658 00010000:00010000:1.0:1785220582.261995:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ee:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.261999:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262006:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262023:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.262039:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+116:2406:x1871938519992448:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (4732us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.262045:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100668 00000100:00100000:2.0:1785220582.262114:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d6540 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.262116:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000018f902d4/0xf62499ab92793404 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e9 expref: 85 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.262128:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8752:1871938510767616:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.262140:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.262144:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262151:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262152:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262160:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262166:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262169:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262173:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262177:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262180:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262193:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220582.262196:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.262292:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c682300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1340:3655:x1871938510764800:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24766us (26060us total) trans 4295070190 rc 0/0 00000100:00100000:1.0:1785220582.262292:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+146:3653:x1871938510765824:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16799us (17617us total) trans 4295070767 rc 0/0 00000100:00100000:2.0:1785220582.262299:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284721 00000100:00100000:1.0:1785220582.262302:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284730 00000100:00100000:2.0:1785220582.262331:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+115:2406:x1871938519991680:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.262332:0:14164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fba1fe8e/0xf62499ab92792ca2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6f8 expref: 115 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262340:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.262342:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0001_UUID:14164:1871938510768000:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.262345:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d1a469f3/0xf62499ab92792adb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ce expref: 114 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262353:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262354:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d1a469f3/0xf62499ab92792adb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ce expref: 114 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262361:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262368:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d1a469f3/0xf62499ab92792adb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f6ce expref: 114 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262369:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262378:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.262378:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262381:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ec:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.262386:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262386:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262395:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262405:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220582.262407:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.262411:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262418:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.262421:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262429:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262439:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262445:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cbb6036/0xf62499ab927a5844 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ea:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262460:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.262464:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.262469:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+1340:3655:x1871938510766592:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16767us (17087us total) trans 4295070191 rc 0/0 00010000:00010000:2.0:1785220582.262470:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.262474:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.262475:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284732 00010000:00010000:2.0:1785220582.262486:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262496:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262502:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d120fb0/0xf62499ab927a587c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262514:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.262516:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262522:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.262524:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262531:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262534:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262540:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262542:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262548:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a588a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ef:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262551:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262559:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262565:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.262566:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.262567:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.262568:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.262571:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262577:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.262579:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262586:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262595:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262602:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007e23c80f/0xf62499ab927a5883 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262614:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.262616:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.262617:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.262629:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+146:3653:x1871938510766848:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17071us (17920us total) trans 4295070768 rc 0/0 00010000:00010000:2.0:1785220582.262631:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.262638:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284731 00000100:00100000:2.0:1785220582.262648:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9083f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+113:2406:x1871938519991680:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2229us (5452us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.262655:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100665 00010000:00010000:1.0:1785220582.262657:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262665:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262672:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262678:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262695:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.262746:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.262761:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.262769:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.262772:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.262773:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8601f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+146:3653:x1871938510765440:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18356us (18502us total) trans 4295070769 rc 0/0 00000100:00100000:1.0:1785220582.262780:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284726 00010000:00010000:3.0:1785220582.262781:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.262790:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.262797:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5860 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262801:0:14145:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262808:0:14145:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.262810:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.262812:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.262813:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.262817:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.262817:0:14145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262830:0:14145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.262846:0:14145:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.262915:0:14145:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+1340:3655:x1871938510766464:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16599us (16951us total) trans 4295070192 rc 0/0 00000100:00100000:1.0:1785220582.262922:0:14145:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284734 00010000:00010000:2.0:1785220582.262964:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.262969:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262976:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922acc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.262979:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262986:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.262996:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.263003:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fa7c81c0/0xf62499ab927a5898 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ee:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.263016:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.263018:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.263020:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.263552:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.263577:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93763b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510768768:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.263595:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.263622:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93761500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510768896:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.263633:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.263720:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+145:3655:x1871938510767104:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17274us (17719us total) trans 4295070766 rc 0/0 00000100:00100000:0.0:1785220582.263727:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284735 00010000:00010000:0.0:1785220582.263759:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.263768:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.263776:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.263783:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.263880:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.263974:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1340:3653:x1871938510765312:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18537us (19331us total) trans 4295070193 rc 0/0 00000100:00100000:0.0:1785220582.263981:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284729 00010000:00010000:0.0:1785220582.264003:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.264014:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.264023:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.264035:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264067:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.264072:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264079:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ac000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.264082:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264119:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264141:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264149:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000270ea293/0xf62499ab927a5875 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ec:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264178:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.264182:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264195:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d922ad200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.264198:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264205:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264216:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.264222:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.264224:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.264226:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.264336:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.264412:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1785220582.264417:0:14178:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062842ed7/0xf62499ab927a5891 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.264504:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90cbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+1340:3655:x1871938510767872:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18696us (18757us total) trans 4295070194 rc 0/0 00000004:00080000:1.0:1785220582.264505:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724000/2, rc 0, transno 4295070193 00000100:00100000:0.0:1785220582.264511:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284733 00000100:00100000:1.0:1785220582.264527:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510765312:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.264541:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9083e680/2, rc 0, transno 4295070767 00000100:00100000:1.0:1785220582.264560:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9083e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510765824:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.264572:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d310700/2, rc 0, transno 4295070768 00000100:00100000:1.0:1785220582.264575:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d310700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510766848:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.264583:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8b108700/2, rc 0, transno 4295070769 00000100:00100000:1.0:1785220582.264586:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8b108700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510765440:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.264613:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220582.264626:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93761500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510770304:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.264660:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.264676:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94dbd880/2, rc 0, transno 4295070192 00000100:00100000:1.0:1785220582.264688:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94dbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510766464:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.264701:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510770304 00000100:00100000:1.0:1785220582.264705:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510770304 00000100:00100000:1.0:1785220582.264743:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284743 00000100:00100000:0.0:1785220582.264796:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510770688:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.264889:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.264913:0:14209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510770688 00000100:00100000:0.0:1785220582.264916:0:14209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510770688 00000100:00100000:0.0:1785220582.264941:0:14209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284744 00000100:00100000:0.0:1785220582.264947:0:14209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+145:3654:x1871938510770688:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.264971:0:14209:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29468 count 1 00010000:00010000:0.0:1785220582.264992:0:14209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.265007:0:14209:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000acb860a1/0xf62499ab92793482 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f967 expref: 84 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.265015:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x731c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.265019:0:14209:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.265023:0:14209:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.265045:0:14209:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000acb860a1/0xf62499ab92793482 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f967 expref: 84 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.265057:0:14209:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000acb860a1/0xf62499ab92793482 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f967 expref: 84 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.265069:0:14209:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510770944:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.265226:0:14209:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.265244:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.265250:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510770816:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.265273:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.265298:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510770432:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.265322:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.265342:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220582.265347:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510770560:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.265367:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.265437:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510770816 00000100:00100000:3.0:1785220582.265441:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510770816 00000100:00100000:3.0:1785220582.265472:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510770432 00000100:00100000:3.0:1785220582.265474:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510770432 00000100:00100000:3.0:1785220582.265488:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510770560 00000100:00100000:3.0:1785220582.265491:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510770560 00000100:00100000:3.0:1785220582.265506:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284745 00000100:00100000:3.0:1785220582.265512:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+147:3654:x1871938510770816:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.265525:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29469 count 1 00010000:00010000:3.0:1785220582.265538:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265551:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000629cce30/0xf62499ab92793490 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f975 expref: 84 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265558:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x731d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.265561:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.265565:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 4/0,1 mode: --/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265583:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000629cce30/0xf62499ab92793490 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f975 expref: 84 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265592:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000629cce30/0xf62499ab92793490 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f975 expref: 84 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.265602:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93760000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510771072:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.265626:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.265643:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284746 00000100:00100000:3.0:1785220582.265651:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1341:3652:x1871938510770432:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.265663:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29429 count 1 00010000:00010000:3.0:1785220582.265676:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265687:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d1a10edf/0xf62499ab927930fb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f74c expref: 112 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265699:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f5:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.265702:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.265706:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265725:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d1a10edf/0xf62499ab927930fb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f74c expref: 112 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.265735:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d1a10edf/0xf62499ab927930fb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f74c expref: 112 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.265738:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1341:3654:x1871938510770304:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.265746:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93763800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510771200:192.168.201.51@tcp:104: 00002000:00080000:1.0:1785220582.265762:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29428 count 1 00010000:00010000:1.0:1785220582.265774:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.265783:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.265787:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000016dd4e6d/0xf62499ab9279303e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f73e expref: 112 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.265793:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f4:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.265796:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.265799:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.265818:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000016dd4e6d/0xf62499ab9279303e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f73e expref: 112 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.265828:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000016dd4e6d/0xf62499ab9279303e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f73e expref: 112 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.265837:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510771328:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.265861:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a93c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.265878:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284747 00000100:00100000:1.0:1785220582.265886:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90141500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+147:3652:x1871938510770560:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.265898:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29467 count 1 00010000:00010000:1.0:1785220582.265909:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.265921:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000013424c5e/0xf62499ab92793474 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f959 expref: 84 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.265932:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x731b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.265935:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.265939:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.265941:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.265955:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000013424c5e/0xf62499ab92793474 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f959 expref: 84 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.265964:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000013424c5e/0xf62499ab92793474 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f959 expref: 84 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.265972:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510771456:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.265993:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9780 going to sleep for 11 seconds 00000004:00080000:1.0:1785220582.266031:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d313480/2, rc 0, transno 4295070766 00000100:00100000:1.0:1785220582.266049:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d313480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510767104:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.266062:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbd880/2, rc 0, transno 4295070194 00000100:00100000:1.0:1785220582.266075:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510767872:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.266390:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d312300/2, rc 0, transno 4295070191 00000100:00100000:1.0:1785220582.266395:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510766592:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.266408:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0ce00/2, rc 0, transno 4295070190 00000100:00100000:1.0:1785220582.266411:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510764800:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.266420:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.266430:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90141880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510771584:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.266458:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.266493:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510771584 00000100:00100000:1.0:1785220582.266498:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510771584 00000100:00100000:1.0:1785220582.266533:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284748 00000100:00100000:1.0:1785220582.266540:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+148:3653:x1871938510771584:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.266551:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29470 count 1 00010000:00010000:1.0:1785220582.266564:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.266577:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009a12a3bf/0xf62499ab9279349e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f983 expref: 84 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.266584:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x731e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.266587:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.266590:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 4/0,1 mode: --/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.266672:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93762a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510771840:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.266677:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009a12a3bf/0xf62499ab9279349e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f983 expref: 84 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.266688:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009a12a3bf/0xf62499ab9279349e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f983 expref: 84 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.266696:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.266698:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510771712:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.266730:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9000 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.266742:0:13818:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510771840 00000100:00100000:2.0:1785220582.266746:0:13818:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510771840 00000100:00100000:2.0:1785220582.266787:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284749 00000100:00100000:2.0:1785220582.266795:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90141180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1342:3653:x1871938510771840:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.266806:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29430 count 1 00010000:00010000:2.0:1785220582.266818:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.266840:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006c73712d/0xf62499ab92793172 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f75a expref: 112 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.266847:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f6:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.266850:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.266854:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.266874:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006c73712d/0xf62499ab92793172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f75a expref: 112 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.266883:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006c73712d/0xf62499ab92793172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f75a expref: 112 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.266896:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510771968:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.267292:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510772224:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.267292:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93760e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510772096:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.267317:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.267335:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510772096 00000100:00100000:1.0:1785220582.267339:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510772096 00000100:00100000:0.0:1785220582.267350:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.267370:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510772224 00000100:00100000:1.0:1785220582.267372:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284750 00000100:00100000:0.0:1785220582.267372:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510772224 00000100:00100000:1.0:1785220582.267378:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1344:3654:x1871938510772096:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.267390:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29431 count 1 00000100:00100000:0.0:1785220582.267392:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284751 00000100:00100000:0.0:1785220582.267398:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1344:3655:x1871938510772224:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.267401:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.267406:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29432 count 1 00010000:00010000:1.0:1785220582.267414:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd877559/0xf62499ab92793205 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f768 expref: 112 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.267417:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.267422:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f7:0x0].0x0 since=18s v=(0 0) 00010000:00010000:1.0:1785220582.267425:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.267428:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000082d190b7/0xf62499ab9279326e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f776 expref: 112 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.267429:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.267435:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f8:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.267438:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.267441:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.267448:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd877559/0xf62499ab92793205 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f768 expref: 112 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.267455:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000082d190b7/0xf62499ab9279326e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f776 expref: 112 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.267458:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd877559/0xf62499ab92793205 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f768 expref: 112 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.267465:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000082d190b7/0xf62499ab9279326e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f776 expref: 112 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.267468:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510772352:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.267473:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510772480:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.267497:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9480 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.267530:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.269113:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.269208:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992832 00000100:00100000:3.0:1785220582.269215:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992832 00010000:00010000:3.0:1785220582.269248:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000805a98e9/0xf62499ab92792e1c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f714 expref: 113 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.269275:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100669 00000100:00100000:3.0:1785220582.269285:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+113:2406:x1871938519992832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.269298:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.269304:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000805a98e9/0xf62499ab92792e1c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f714 expref: 113 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269323:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000805a98e9/0xf62499ab92792e1c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f714 expref: 113 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269345:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000805a98e9/0xf62499ab92792e1c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f714 expref: 113 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269363:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.269367:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.269373:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269384:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269408:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.269449:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+112:2406:x1871938519992832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 166us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.269457:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100669 00010000:00010000:3.0:1785220582.269487:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269498:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269507:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269515:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269531:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.269688:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1345:3653:x1871938510768640:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14178us (15750us total) trans 4295070195 rc 0/0 00000100:00100000:3.0:1785220582.269695:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284737 00010000:00010000:3.0:1785220582.269715:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.269721:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269730:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.269734:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269743:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269767:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269774:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d55836b/0xf62499ab927a58ad lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.269804:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.269806:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.269808:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.270039:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.270537:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992704 00000100:00100000:0.0:1785220582.270541:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.270543:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992704 00010000:00010000:1.0:1785220582.270566:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001603306c/0xf62499ab9279343c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f921 expref: 85 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.270597:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100670 00000100:00100000:1.0:1785220582.270606:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+85:2406:x1871938519992704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.270629:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.270634:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001603306c/0xf62499ab9279343c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f921 expref: 85 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.270644:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001603306c/0xf62499ab9279343c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f921 expref: 85 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.270658:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001603306c/0xf62499ab9279343c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f921 expref: 85 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.270671:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7317:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.270674:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7317:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.270678:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.270686:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.270714:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.270736:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+84:2406:x1871938519992704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 131us (698us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.270742:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.270743:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100670 00010000:00010000:0.0:1785220582.270761:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.270771:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.270778:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.270799:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efd180/2, rc 0, transno 4295070195 00000100:00100000:1.0:1785220582.270812:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510768640:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.270830:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.270871:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510772608:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.270905:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.270954:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+149:3653:x1871938510768512:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15695us (17177us total) trans 4295070770 rc 0/0 00000100:00100000:0.0:1785220582.270962:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284736 00000004:00080000:1.0:1785220582.270964:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efe680/2, rc 0, transno 4295070770 00000100:00100000:1.0:1785220582.270973:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510768512:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.270975:0:14204:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510772608 00000100:00100000:0.0:1785220582.270978:0:14204:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510772608 00000100:00100000:0.0:1785220582.271008:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284752 00000100:00100000:0.0:1785220582.271014:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1345:3652:x1871938510772608:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.271033:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29433 count 1 00010000:00010000:0.0:1785220582.271051:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.271064:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004d62c053/0xf62499ab9279329f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f784 expref: 111 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.271076:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72f9:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.271078:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.271082:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.271228:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004d62c053/0xf62499ab9279329f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f784 expref: 111 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.271242:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004d62c053/0xf62499ab9279329f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f784 expref: 111 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.271254:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510772736:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.271281:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.271324:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220582.271332:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510772864:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.271356:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.271973:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.271981:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.271990:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.271994:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.272004:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.272020:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.272028:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a58a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.272191:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.272193:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510772864 00010000:00010000:2.0:1785220582.272194:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.272196:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.272199:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510772864 00000100:00100000:1.0:1785220582.272238:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284753 00000100:00100000:1.0:1785220582.272246:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+149:3652:x1871938510772864:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.272263:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29471 count 1 00010000:00010000:1.0:1785220582.272277:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.272292:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0befd3f/0xf62499ab927934ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f991 expref: 83 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.272301:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x731f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.272304:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.272309:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.272331:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e0befd3f/0xf62499ab927934ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f991 expref: 83 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.272342:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e0befd3f/0xf62499ab927934ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f991 expref: 83 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.272354:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510772992:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.272402:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8c1a9c00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.274454:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.274480:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0000_UUID:14202:1871938510769536:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.274484:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.274500:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.274510:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.274529:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.274679:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93760000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510771072:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.274699:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.274785:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93763800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510771200:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.274801:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.274874:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.274937:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98addc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510769408:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.274952:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.274981:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0000_UUID:14266:1871938510770048:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.274994:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.275021:0:14209:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_065:lustre-OST0000_UUID:14209:1871938510770944:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.275032:0:14209:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.275054:0:14232:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_071:lustre-OST0000_UUID:14232:1871938510769664:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.275065:0:14232:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.275542:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.275602:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519992960 00000100:00100000:1.0:1785220582.275733:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519992960 00010000:00010000:1.0:1785220582.275778:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005edbefcf/0xf62499ab92792ecb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f722 expref: 112 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.275799:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993216 00000100:00100000:0.0:1785220582.275803:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993216 00000100:00100000:1.0:1785220582.275808:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993088 00000100:00100000:1.0:1785220582.275810:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993088 00010000:00010000:1.0:1785220582.275820:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003c9b1979/0xf62499ab92793458 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f93d expref: 85 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.275829:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009c7a193c/0xf62499ab9279344a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f92f expref: 85 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.275837:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993344 00000100:00100000:1.0:1785220582.275840:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993344 00000100:00100000:0.0:1785220582.275846:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519994368 00000100:00100000:0.0:1785220582.275849:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519994368 00010000:00010000:1.0:1785220582.275851:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000720211d4/0xf62499ab92793466 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f94b expref: 86 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.275859:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009a12a3bf/0xf62499ab9279349e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f983 expref: 87 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.275869:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100671 00000100:00100000:1.0:1785220582.275877:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+112:2404:x1871938519992960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.275891:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.275898:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005edbefcf/0xf62499ab92792ecb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f722 expref: 112 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.275910:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005edbefcf/0xf62499ab92792ecb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f722 expref: 112 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.275932:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005edbefcf/0xf62499ab92792ecb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f722 expref: 112 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.275952:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.275957:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f2:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.275963:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.275968:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.275972:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.275985:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.275994:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.276000:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519994240 00000100:00100000:3.0:1785220582.276003:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519994240 00010000:00010000:3.0:1785220582.276018:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000013424c5e/0xf62499ab92793474 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f959 expref: 88 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.276036:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97841180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+111:2404:x1871938519992960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 160us (1592us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.276061:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993472 00000100:00100000:0.0:1785220582.276064:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993472 00010000:00010000:0.0:1785220582.276081:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd877559/0xf62499ab92793205 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f768 expref: 112 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.276175:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993728 00000100:00100000:0.0:1785220582.276179:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993728 00010000:00010000:0.0:1785220582.276194:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003dd71c0d/0xf62499ab92792fc7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f730 expref: 113 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.276214:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100672 00000100:00100000:0.0:1785220582.276220:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90700050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+88:2404:x1871938519993088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.276229:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.276234:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003c9b1979/0xf62499ab92793458 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f93d expref: 88 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276242:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003c9b1979/0xf62499ab92793458 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f93d expref: 88 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276255:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c9b1979/0xf62499ab92793458 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f93d expref: 88 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276264:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7319:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276267:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7319:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276271:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276279:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276297:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.276316:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90700050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+87:2404:x1871938519993088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1815us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.276324:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100672 00000100:00100000:0.0:1785220582.276340:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100673 00000100:00100000:0.0:1785220582.276346:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+86:2404:x1871938519993216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.276352:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.276356:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009c7a193c/0xf62499ab9279344a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f92f expref: 86 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276364:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009c7a193c/0xf62499ab9279344a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f92f expref: 86 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276376:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c7a193c/0xf62499ab9279344a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f92f expref: 86 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276387:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7318:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276390:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7318:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276393:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276400:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276416:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.276434:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97842300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+85:2404:x1871938519993216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (1948us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.276440:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100673 00000100:00100000:0.0:1785220582.276455:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100674 00000100:00100000:0.0:1785220582.276460:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+84:2405:x1871938519993344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.276467:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.276471:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000720211d4/0xf62499ab92793466 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f94b expref: 84 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276480:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000720211d4/0xf62499ab92793466 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f94b expref: 84 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276492:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000720211d4/0xf62499ab92793466 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f94b expref: 84 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276500:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276502:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276506:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276513:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276528:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.276546:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97840e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+83:2405:x1871938519993344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 85us (2015us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.276552:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100674 00000100:00100000:0.0:1785220582.276564:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100675 00000100:00100000:0.0:1785220582.276570:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97843b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+82:2405:x1871938519994368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.276578:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.276583:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009a12a3bf/0xf62499ab9279349e lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f983 expref: 82 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276591:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009a12a3bf/0xf62499ab9279349e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f983 expref: 82 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276602:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276605:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276610:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276617:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276629:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.276646:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97843b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+82:2405:x1871938519994368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (1772us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.276653:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100675 00000100:00100000:0.0:1785220582.276666:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100676 00000100:00100000:0.0:1785220582.276672:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+113:2407:x1871938519993472:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.276679:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.276684:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd877559/0xf62499ab92793205 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f768 expref: 113 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276698:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd877559/0xf62499ab92793205 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f768 expref: 113 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276711:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276714:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.276719:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276727:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276739:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.276770:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+113:2407:x1871938519993472:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (801us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.276776:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100676 00000100:00100000:0.0:1785220582.276789:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100677 00000100:00100000:0.0:1785220582.276796:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+112:2407:x1871938519993728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.276802:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.276806:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003dd71c0d/0xf62499ab92792fc7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f730 expref: 112 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.276816:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003dd71c0d/0xf62499ab92792fc7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f730 expref: 112 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277168:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.277172:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.277177:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277187:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277202:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.277241:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+112:2407:x1871938519993728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 445us (1254us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.277247:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100677 00000100:00100000:2.0:1785220582.277303:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510771968:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.277322:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277356:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277364:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277373:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277379:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277395:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.277549:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+150:3653:x1871938510769792:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18073us (18129us total) trans 4295070771 rc 0/0 00000100:00100000:0.0:1785220582.277557:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284741 00010000:00010000:0.0:1785220582.277574:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277582:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277590:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277596:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277606:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.277701:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98ade680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+1346:3653:x1871938510769280:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20109us (20164us total) trans 4295070196 rc 0/0 00000100:00100000:0.0:1785220582.277707:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284738 00010000:00010000:2.0:1785220582.277766:0:13866:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003dd71c0d/0xf62499ab92792fc7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f730 expref: 111 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.277776:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510770176:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.277784:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.277790:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.277794:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.277799:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.277802:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.277813:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1785220582.277877:0:14232:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277883:0:14232:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277888:0:14232:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277896:0:14232:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.277905:0:14232:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220582.277960:0:14232:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98ade300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_071:lustre-MDT0000-mdtlov_UUID+150:3655:x1871938510769024:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19327us (19905us total) trans 4295070772 rc 0/0 00000100:00100000:0.0:1785220582.277964:0:14232:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284740 00010000:00010000:0.0:1785220582.277987:0:14194:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd877559/0xf62499ab92793205 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f768 expref: 110 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.277993:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510772352:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.278002:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.278008:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.278011:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.278016:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.278020:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.278040:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:2.0:1785220582.278273:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+1347:3655:x1871938510769920:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17771us (18046us total) trans 4295070198 rc 0/0 00000100:00100000:2.0:1785220582.278282:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284742 00010000:00010000:2.0:1785220582.278302:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1785220582.278307:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278320:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.278324:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278332:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278344:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278354:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d098f110/0xf62499ab927a58c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278371:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220582.278374:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278380:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4da00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.278382:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278389:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278397:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278404:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ecdc5d08/0xf62499ab927a58b4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278414:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.278417:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278422:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.278425:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278432:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278439:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278444:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ab24a30d/0xf62499ab927a58d0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278454:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.278457:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278462:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.278465:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278472:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278479:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278485:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a455e97/0xf62499ab927a58c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14232 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278495:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.278497:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278502:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.278505:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278511:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278518:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278524:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dc7c75e7/0xf62499ab927a5908 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.278539:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90140380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+1346:3654:x1871938510772096:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10741us (10801us total) trans 4295070197 rc 0/0 00000100:00100000:0.0:1785220582.278545:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284750 00000100:00100000:0.0:1785220582.278649:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510771328:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.278664:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.278817:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100671 00000100:00100000:1.0:1785220582.278865:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100678 00000100:00100000:1.0:1785220582.278875:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+81:2405:x1871938519994240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.278888:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.278894:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000013424c5e/0xf62499ab92793474 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f959 expref: 81 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278895:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.278897:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.278898:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.278906:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.278910:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000013424c5e/0xf62499ab92793474 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f959 expref: 81 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.278911:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.278926:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.278930:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.278935:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.278945:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.278950:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.278959:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.278985:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+81:2405:x1871938519994240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (3444us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.278990:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100678 00000004:00080000:0.0:1785220582.278990:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93760e00/2, rc 0, transno 4295070197 00000100:00100000:0.0:1785220582.279003:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93760e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510772096:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.279026:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98adca80/2, rc 0, transno 4295070771 00000100:00100000:0.0:1785220582.279034:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510769792:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.279078:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510773120:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.279245:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.279308:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.279339:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.279353:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993984 00000100:00100000:2.0:1785220582.279358:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993984 00010000:00010000:2.0:1785220582.279379:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000629cce30/0xf62499ab92793490 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f975 expref: 81 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.279398:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519994496 00000100:00100000:2.0:1785220582.279400:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519994496 00010000:00010000:2.0:1785220582.279411:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000016dd4e6d/0xf62499ab9279303e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f73e expref: 109 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.279421:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993600 00000100:00100000:2.0:1785220582.279423:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993600 00010000:00010000:2.0:1785220582.279432:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006c73712d/0xf62499ab92793172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f75a expref: 110 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.279441:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100679 00000100:00100000:2.0:1785220582.279447:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+81:2407:x1871938519993984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.279455:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.279460:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000629cce30/0xf62499ab92793490 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f975 expref: 81 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.279518:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100680 00000100:00100000:1.0:1785220582.279525:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+110:2407:x1871938519994496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.279533:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100681 00010000:00010000:1.0:1785220582.279534:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.279538:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000016dd4e6d/0xf62499ab9279303e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f73e expref: 110 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.279542:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+110:2407:x1871938519993600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.279548:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.279548:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000016dd4e6d/0xf62499ab9279303e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f73e expref: 110 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279553:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006c73712d/0xf62499ab92793172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f75a expref: 110 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279563:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006c73712d/0xf62499ab92793172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f75a expref: 110 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.279568:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016dd4e6d/0xf62499ab9279303e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f73e expref: 110 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279574:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006c73712d/0xf62499ab92793172 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f75a expref: 110 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.279578:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.279582:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.279586:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279588:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.279591:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.279594:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279595:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279603:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279617:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.279639:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+108:2407:x1871938519993600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.279645:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100681 00000100:00100000:3.0:1785220582.279682:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.279683:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.279696:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519993856 00000100:00100000:3.0:1785220582.279699:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519993856 00000100:00100000:1.0:1785220582.279702:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+107:2407:x1871938519994496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 178us (393us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.279709:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100680 00010000:00010000:3.0:1785220582.279713:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000acb860a1/0xf62499ab92793482 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f967 expref: 82 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.279730:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100682 00000100:00100000:3.0:1785220582.279735:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+82:2407:x1871938519993856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.279740:0:14159:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000013424c5e/0xf62499ab92793474 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f959 expref: 82 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279743:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.279747:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000acb860a1/0xf62499ab92793482 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f967 expref: 82 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.279764:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510771456:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.279774:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000acb860a1/0xf62499ab92793482 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f967 expref: 81 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.279777:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.279786:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279788:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb860a1/0xf62499ab92793482 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f967 expref: 81 pid: 14207 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279798:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.279799:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279801:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.279805:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.279808:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.279813:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.279817:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.279829:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.279891:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.279925:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.279935:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.279941:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90141500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+150:3652:x1871938510770560:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14056us (14573us total) trans 4295070773 rc 0/0 00010000:00010000:0.0:1785220582.279943:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.279948:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284747 00010000:00010000:0.0:1785220582.279955:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.279961:0:14159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510773120 00000100:00100000:1.0:1785220582.279964:0:14159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510773120 00010000:00010000:0.0:1785220582.279968:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.279983:0:14159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284754 00000100:00100000:1.0:1785220582.279989:0:14159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+150:3654:x1871938510773120:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.280007:0:14159:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29472 count 1 00010000:00010000:1.0:1785220582.280024:0:14159:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280036:0:14159:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000082d294a6/0xf62499ab927934ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f99f expref: 80 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.280041:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1346:3654:x1871938510770304:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15291us (15381us total) trans 4295070199 rc 0/0 00010000:00010000:1.0:1785220582.280044:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7320:0x0].0x0 since=17s v=(0 0) 00000100:00100000:0.0:1785220582.280047:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284743 00010000:00010000:1.0:1785220582.280048:0:14159:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7320:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.280051:0:14159:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280074:0:14159:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000082d294a6/0xf62499ab927934ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f99f expref: 80 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.280074:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93761500/2, rc 0, transno 4295070199 00000100:00100000:0.0:1785220582.280079:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93761500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510770304:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.280215:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000629cce30/0xf62499ab92793490 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f975 expref: 80 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280229:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000629cce30/0xf62499ab92793490 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f975 expref: 80 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280239:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.280242:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.280246:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280252:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280271:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.280282:0:14159:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000082d294a6/0xf62499ab927934ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f99f expref: 79 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.280288:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+79:2407:x1871938519993984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 841us (1044us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.280294:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100679 00000100:00100000:1.0:1785220582.280296:0:14159:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510773248:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.280327:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.280328:0:14159:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.280336:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280349:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280349:0:13908:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a12a3bf/0xf62499ab9279349e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f983 expref: 78 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280357:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.280360:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:13908:1871938510771712:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.280369:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220582.280372:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280379:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280386:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280399:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280405:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280422:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:2.0:1785220582.280439:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90141180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1346:3653:x1871938510771840:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13645us (13742us total) trans 4295070200 rc 0/0 00000100:00100000:2.0:1785220582.280446:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284749 00010000:00010000:2.0:1785220582.280462:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220582.280467:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280475:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.280478:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280487:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.280488:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90140700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+151:3653:x1871938510771584:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13949us (14029us total) trans 4295070774 rc 0/0 00000100:00100000:1.0:1785220582.280494:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284748 00010000:00010000:2.0:1785220582.280500:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280507:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a58f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280521:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.280525:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280531:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.280532:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280533:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280540:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280542:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280548:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280551:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280556:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280558:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a58ec lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.280565:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:2.0:1785220582.280572:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.280575:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280582:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.280584:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280596:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280606:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280612:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5901 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280633:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.280638:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280644:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.280647:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280654:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280665:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 1/0,0 mode: --/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280672:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041eeac35/0xf62499ab927a58fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280690:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.280693:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.280696:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d935a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+151:3655:x1871938510769152:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22809us (22857us total) trans 4295070775 rc 0/0 00010000:00010000:2.0:1785220582.280700:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.280703:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.280706:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284739 00010000:00010000:2.0:1785220582.280710:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280719:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280725:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000399613ec/0xf62499ab927a58bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.280736:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.280738:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.280739:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.280745:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.280761:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.280813:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:0.0:1785220582.280899:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98adc700/2, rc 0, transno 4295070196 00010000:00010000:3.0:1785220582.280956:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.280972:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+77:2407:x1871938519993856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1237us (1290us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.280979:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100682 00000100:00100000:2.0:1785220582.281054:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.281082:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98adc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510769280:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.281103:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:0.0:1785220582.281114:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.281133:0:14209:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281143:0:14209:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281151:0:14209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281158:0:14209:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281181:0:14209:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.281262:0:14209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_065:lustre-MDT0000-mdtlov_UUID+151:3654:x1871938510770688:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16311us (16436us total) trans 4295070776 rc 0/0 00000100:00100000:0.0:1785220582.281268:0:14209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284744 00000100:00100000:0.0:1785220582.281294:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510772480:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.281307:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.281343:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4dc00/2, rc 0, transno 4295070776 00000100:00100000:0.0:1785220582.281351:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510770688:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.281355:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.281379:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510773376:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.281399:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.281417:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510773376 00000100:00100000:0.0:1785220582.281421:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510773376 00000100:00100000:0.0:1785220582.281444:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284755 00000100:00100000:0.0:1785220582.281450:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d955fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1346:3654:x1871938510773376:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.281464:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29434 count 1 00010000:00010000:0.0:1785220582.281548:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281560:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000b42445f/0xf62499ab927932ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f792 expref: 106 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281573:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72fa:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.281576:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.281580:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281598:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000b42445f/0xf62499ab927932ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f792 expref: 106 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281610:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000b42445f/0xf62499ab927932ad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f792 expref: 106 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.281622:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510773888:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.281650:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.281670:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510773760:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.281690:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.281705:0:14223:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510773760 00000100:00100000:0.0:1785220582.281708:0:14223:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510773760 00000100:00100000:0.0:1785220582.281722:0:14223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284756 00000100:00100000:0.0:1785220582.281726:0:14223:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+151:3654:x1871938510773760:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.281734:0:14223:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29473 count 1 00010000:00010000:0.0:1785220582.281743:0:14223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281765:0:14223:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000071a5ad15/0xf62499ab927934c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ad expref: 76 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281773:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7321:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.281776:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7321:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.281783:0:14223:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281803:0:14223:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000071a5ad15/0xf62499ab927934c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ad expref: 76 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.281813:0:14223:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000071a5ad15/0xf62499ab927934c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ad expref: 76 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.281824:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519994112 00000100:00100000:1.0:1785220582.281829:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519994112 00010000:00010000:1.0:1785220582.281849:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d1a10edf/0xf62499ab927930fb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f74c expref: 107 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.281872:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100683 00000100:00100000:1.0:1785220582.281879:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+107:2407:x1871938519994112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.281888:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.281893:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d1a10edf/0xf62499ab927930fb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f74c expref: 107 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.281893:0:14223:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0000_UUID:14223:1871938510774016:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.281904:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d1a10edf/0xf62499ab927930fb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f74c expref: 107 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.281911:0:14223:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.281919:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d1a10edf/0xf62499ab927930fb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f74c expref: 107 pid: 14201 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.281930:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.281933:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.281938:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.281946:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.281948:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4fb80/2, rc 0, transno 4295070773 00000100:00100000:0.0:1785220582.281956:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510770560:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.281969:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.281984:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2407:x1871938519994112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (1171us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.281984:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.281990:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91479880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510774144:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.281991:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100683 00000100:00100000:0.0:1785220582.282013:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.282038:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510774144 00000100:00100000:0.0:1785220582.282038:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.282041:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510774144 00000100:00100000:0.0:1785220582.282045:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510773632:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.282063:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284757 00000100:00100000:0.0:1785220582.282066:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.282069:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91478700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+152:3654:x1871938510774144:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.282080:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29474 count 1 00000004:00080000:0.0:1785220582.282169:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4c380/2, rc 0, transno 4295070198 00000100:00100000:0.0:1785220582.282181:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510769920:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.282198:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282207:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282212:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9147b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510774272:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.282216:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282224:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282233:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.282238:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1785220582.282262:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90cbf480/2, rc 0, transno 4295070772 00000100:00100000:0.0:1785220582.282270:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90cbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510769024:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.282293:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220582.282300:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9147b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510774400:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.282319:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.282328:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+153:3654:x1871938510770816:12345-0@lo:6:osp-syn-0-0.0 Request processed in 16817us (17054us total) trans 4295070777 rc 0/0 00000100:00100000:3.0:1785220582.282334:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284745 00000004:00080000:0.0:1785220582.282345:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4df80/2, rc 0, transno 4295070777 00000100:00100000:3.0:1785220582.282349:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510773632 00000100:00100000:0.0:1785220582.282351:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510770816:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.282352:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510773632 00000100:00100000:3.0:1785220582.282369:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510774272 00000100:00100000:3.0:1785220582.282372:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510774272 00000100:00100000:0.0:1785220582.282379:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c92a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510774528:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.282387:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510774400 00000100:00100000:3.0:1785220582.282389:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510774400 00000100:00100000:0.0:1785220582.282399:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.282405:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510774528 00000100:00100000:3.0:1785220582.282407:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510774528 00000100:00100000:3.0:1785220582.282424:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284758 00000004:00080000:0.0:1785220582.282426:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97841880/2, rc 0, transno 4295070775 00000100:00100000:3.0:1785220582.282430:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9147bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1348:3652:x1871938510773632:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.282432:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97841880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510769152:0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.282441:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29436 count 1 00010000:00010000:3.0:1785220582.282453:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282455:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.282461:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c928380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510774656:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.282465:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000014a11cd6/0xf62499ab927932c9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ae expref: 105 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282472:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72fc:0x0].0x0 since=18s v=(0 0) 00010000:00010000:3.0:1785220582.282475:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.282479:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282481:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.282499:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000014a11cd6/0xf62499ab927932c9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ae expref: 105 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282499:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510774656 00000100:00100000:0.0:1785220582.282502:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510774656 00010000:00010000:3.0:1785220582.282509:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000014a11cd6/0xf62499ab927932c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ae expref: 105 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282519:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510774784:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.282521:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284759 00000100:00100000:0.0:1785220582.282526:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1348:3652:x1871938510774272:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.282536:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29437 count 1 00000100:00100000:3.0:1785220582.282543:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.282547:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282556:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.282559:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dae04e73/0xf62499ab927932d7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7bc expref: 105 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282565:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.282566:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72fd:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.282570:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.282574:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.282574:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282582:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.282592:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dae04e73/0xf62499ab927932d7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7bc expref: 105 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282594:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1785220582.282601:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dae04e73/0xf62499ab927932d7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7bc expref: 105 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282610:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510774912:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.282627:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.282642:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284760 00000100:00100000:0.0:1785220582.282649:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9147a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+155:3652:x1871938510774400:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.282661:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29475 count 1 00010000:00010000:0.0:1785220582.282672:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.282683:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c02efd77/0xf62499ab927934e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c9 expref: 76 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282690:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+1349:3652:x1871938510770432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17041us (17367us total) trans 4295070201 rc 0/0 00010000:00010000:0.0:1785220582.282690:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7323:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.282693:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7323:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.282697:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284746 00010000:00010000:0.0:1785220582.282697:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282711:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284761 00010000:00010000:0.0:1785220582.282715:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c02efd77/0xf62499ab927934e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c9 expref: 76 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282717:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c928700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+155:3654:x1871938510774528:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.282725:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c02efd77/0xf62499ab927934e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c9 expref: 76 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220582.282727:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29476 count 1 00000100:00100000:0.0:1785220582.282735:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510775040:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.282737:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282746:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000571fa8e8/0xf62499ab927934f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d7 expref: 76 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282763:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.282765:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7324:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.282768:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7324:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.282772:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.282787:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93761180/2, rc 0, transno 4295070201 00010000:00010000:3.0:1785220582.282791:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000571fa8e8/0xf62499ab927934f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d7 expref: 76 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.282795:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510770432:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.282800:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000571fa8e8/0xf62499ab927934f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d7 expref: 76 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282809:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510775168:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.282828:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.282844:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284762 00000100:00100000:3.0:1785220582.282851:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c928a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+155:3654:x1871938510774656:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.282864:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29477 count 1 00010000:00010000:3.0:1785220582.282875:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282886:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000447f9a45/0xf62499ab92793500 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9e5 expref: 76 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282893:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7325:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.282896:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7325:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.282899:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282918:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000447f9a45/0xf62499ab92793500 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9e5 expref: 76 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.282927:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000447f9a45/0xf62499ab92793500 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9e5 expref: 76 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282936:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510775424:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.282938:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.282944:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282953:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.282953:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.282957:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.282965:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.282978:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282983:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519994624 00010000:00010000:2.0:1785220582.282985:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebc9a5d1/0xf62499ab927a58d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.282986:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519994624 00010000:00010000:3.0:1785220582.283002:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000082d190b7/0xf62499ab9279326e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f776 expref: 106 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283005:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.283009:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283015:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.283019:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.283021:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100684 00000100:00100000:3.0:1785220582.283027:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2404:x1871938519994624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.283027:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.283035:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.283036:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 1/0,0 mode: --/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.283039:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000082d190b7/0xf62499ab9279326e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f776 expref: 106 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283044:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c2168cb/0xf62499ab927a58de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.283050:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000082d190b7/0xf62499ab9279326e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f776 expref: 106 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283057:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.283060:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.283062:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000082d190b7/0xf62499ab9279326e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f776 expref: 106 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283067:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.283069:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.283073:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.283076:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.283077:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.283081:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283100:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283108:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a58e5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.283195:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.283197:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.283199:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.283205:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.283210:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.283310:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.283327:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283340:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000110c9113/0xf62499ab927934d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9bb expref: 76 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283347:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7322:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.283350:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7322:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.283354:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283372:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000110c9113/0xf62499ab927934d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9bb expref: 76 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283380:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000110c9113/0xf62499ab927934d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9bb expref: 76 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.283394:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0000_UUID:14172:1871938510775552:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.283416:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.283437:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519994752 00000100:00100000:1.0:1785220582.283440:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519994752 00010000:00010000:1.0:1785220582.283457:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004d62c053/0xf62499ab9279329f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f784 expref: 106 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.283479:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100685 00000100:00100000:1.0:1785220582.283486:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2405:x1871938519994752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.283494:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.283499:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004d62c053/0xf62499ab9279329f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f784 expref: 106 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283507:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004d62c053/0xf62499ab9279329f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f784 expref: 106 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283520:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.283522:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72f9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.283526:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283534:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283552:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.283568:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2405:x1871938519994752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.283574:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100685 00000100:00100000:1.0:1785220582.283606:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510772992:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.283622:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.283644:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.283664:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519994880 00000100:00100000:1.0:1785220582.283668:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519994880 00010000:00010000:1.0:1785220582.283682:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e0befd3f/0xf62499ab927934ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f991 expref: 77 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.283710:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100686 00000100:00100000:1.0:1785220582.283717:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+77:2407:x1871938519994880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.283725:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.283729:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e0befd3f/0xf62499ab927934ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f991 expref: 77 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283739:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e0befd3f/0xf62499ab927934ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f991 expref: 77 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283764:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0befd3f/0xf62499ab927934ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f991 expref: 77 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283780:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.283784:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x731f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.283788:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283797:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283812:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.283830:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2407:x1871938519994880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.283837:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100686 00010000:00010000:1.0:1785220582.283854:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283862:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283871:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283879:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.283892:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.284001:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+156:3652:x1871938510772864:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11755us (12643us total) trans 4295070778 rc 0/0 00000100:00100000:1.0:1785220582.284007:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284753 00000100:00100000:0.0:1785220582.284015:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510775296:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.284045:0:14204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d62c053/0xf62499ab9279329f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f784 expref: 105 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.284056:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0001_UUID:14204:1871938510772736:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.284070:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284078:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284098:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284107:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284114:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284192:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.284269:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+1349:3652:x1871938510772608:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13254us (13363us total) trans 4295070202 rc 0/0 00000100:00100000:0.0:1785220582.284275:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284752 00010000:00010000:0.0:1785220582.284292:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.284297:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284304:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.284307:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284316:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284327:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284334:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f3a0f0/0xf62499ab927a591d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284350:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.284354:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284360:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4ce00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.284364:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284371:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284380:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284387:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eeda96fd/0xf62499ab927a5916 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284401:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.284402:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.284403:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.284427:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510773504:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.284452:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.284484:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93762a00/2, rc 0, transno 4295070200 00000100:00100000:0.0:1785220582.284494:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93762a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510771840:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.284537:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90141880/2, rc 0, transno 4295070774 00000100:00100000:0.0:1785220582.284546:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510771584:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.284586:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510775808:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.284609:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.284627:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510773504 00000100:00100000:0.0:1785220582.284631:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510773504 00000100:00100000:0.0:1785220582.284650:0:14191:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510775808 00000100:00100000:0.0:1785220582.284653:0:14191:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510775808 00000100:00100000:0.0:1785220582.284671:0:14191:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284763 00000100:00100000:0.0:1785220582.284678:0:14191:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1349:3653:x1871938510773504:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.284694:0:14191:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29435 count 1 00010000:00010000:0.0:1785220582.284709:0:14191:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284721:0:14191:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000821f5aeb/0xf62499ab927932bb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a0 expref: 104 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284728:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72fb:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.284732:0:14191:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.284736:0:14191:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284769:0:14191:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000821f5aeb/0xf62499ab927932bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a0 expref: 104 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.284781:0:14191:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000821f5aeb/0xf62499ab927932bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a0 expref: 104 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.284798:0:14191:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510775936:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.284931:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.285024:0:14191:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.285043:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284764 00000100:00100000:0.0:1785220582.285050:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+156:3654:x1871938510775808:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.285063:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29478 count 1 00010000:00010000:0.0:1785220582.285075:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285103:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a1877633/0xf62499ab9279350e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9f3 expref: 75 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285111:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7326:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.285114:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7326:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.285184:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285206:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a1877633/0xf62499ab9279350e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9f3 expref: 75 pid: 14209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285215:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a1877633/0xf62499ab9279350e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9f3 expref: 75 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.285224:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0000_UUID:14205:1871938510776064:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.285248:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.285282:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510775680:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.285305:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.285354:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.285375:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00bb80/2, rc 0, transno 4295070778 00000100:00100000:0.0:1785220582.285384:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510772864:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.285415:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.285421:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510776192:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.285442:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.285461:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510775680 00000100:00100000:0.0:1785220582.285464:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510775680 00000100:00100000:0.0:1785220582.285488:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510775296 00000100:00100000:0.0:1785220582.285491:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510775296 00000100:00100000:0.0:1785220582.285506:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510776192 00000100:00100000:0.0:1785220582.285509:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510776192 00000100:00100000:0.0:1785220582.285528:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284765 00000100:00100000:0.0:1785220582.285535:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1351:3653:x1871938510775680:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.285547:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29439 count 1 00010000:00010000:0.0:1785220582.285559:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285571:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002524ecf5/0xf62499ab927932f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d8 expref: 104 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285578:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72ff:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.285581:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.285585:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285605:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002524ecf5/0xf62499ab927932f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d8 expref: 104 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285614:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002524ecf5/0xf62499ab927932f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d8 expref: 104 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.285624:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510776320:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.285647:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.285662:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284766 00000100:00100000:0.0:1785220582.285669:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1351:3652:x1871938510775296:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.285682:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29438 count 1 00010000:00010000:0.0:1785220582.285693:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285705:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ffdf4c5b/0xf62499ab927932e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ca expref: 104 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285713:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x72fe:0x0].0x0 since=18s v=(0 0) 00010000:00010000:0.0:1785220582.285716:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.285720:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.285738:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ffdf4c5b/0xf62499ab927932e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ca expref: 104 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.286027:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ffdf4c5b/0xf62499ab927932e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ca expref: 104 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.286038:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510776448:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.286063:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.286081:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284767 00000100:00100000:0.0:1785220582.286109:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93608380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+157:3654:x1871938510776192:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.286185:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29479 count 1 00010000:00010000:0.0:1785220582.286199:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.286212:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002ea63c77/0xf62499ab9279351c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa01 expref: 75 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.286219:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7327:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.286222:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7327:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.286226:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.286245:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002ea63c77/0xf62499ab9279351c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa01 expref: 75 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.286254:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002ea63c77/0xf62499ab9279351c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa01 expref: 75 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.286268:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510776576:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.286288:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f300 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.286325:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d998bb480/2, rc 0, transno 4295070202 00000100:00100000:0.0:1785220582.286337:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510772608:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.286373:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.286380:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a98380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510776704:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.286407:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.286501:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510776704 00000100:00100000:2.0:1785220582.286506:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510776704 00000100:00100000:2.0:1785220582.286532:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284768 00000100:00100000:2.0:1785220582.286539:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9573d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1352:3653:x1871938510776704:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.286551:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29440 count 1 00010000:00010000:2.0:1785220582.286563:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.286576:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002df7809b/0xf62499ab92793301 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7e6 expref: 104 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.286584:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7300:0x0].0x0 since=18s v=(0 0) 00010000:00010000:2.0:1785220582.286587:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7300:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.286591:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.286611:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002df7809b/0xf62499ab92793301 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7e6 expref: 104 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.286622:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002df7809b/0xf62499ab92793301 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7e6 expref: 104 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.286633:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510776832:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.286657:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.290494:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.290557:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.290613:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+104:2404:x1871938519994624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 7583us (9556us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.290623:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100684 00010000:00010000:1.0:1785220582.290706:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.290716:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.290727:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.290734:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.290762:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.290923:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1353:3655:x1871938510772224:12345-0@lo:6:osp-syn-1-0.0 Request processed in 23524us (23571us total) trans 4295070203 rc 0/0 00000100:00100000:1.0:1785220582.290931:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284751 00010000:00010000:3.0:1785220582.291316:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.291322:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.291332:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.291336:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.291346:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.291363:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.291370:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ef55399/0xf62499ab927a590f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72f8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.291393:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.291395:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.291397:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.291514:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93761f80/2, rc 0, transno 4295070203 00000100:00100000:1.0:1785220582.291531:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510772224:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.292118:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292140:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292158:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292178:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292193:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292261:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292278:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292298:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.292318:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995008 00000100:00100000:2.0:1785220582.292325:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995008 00010000:00010000:2.0:1785220582.292355:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000b42445f/0xf62499ab927932ad lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f792 expref: 104 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292383:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995264 00000100:00100000:2.0:1785220582.292386:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995264 00010000:00010000:2.0:1785220582.292399:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000014a11cd6/0xf62499ab927932c9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ae expref: 105 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292418:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995520 00000100:00100000:2.0:1785220582.292421:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995520 00010000:00010000:2.0:1785220582.292435:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000110c9113/0xf62499ab927934d6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9bb expref: 76 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292452:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995136 00000100:00100000:2.0:1785220582.292455:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995136 00010000:00010000:2.0:1785220582.292467:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000071a5ad15/0xf62499ab927934c8 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ad expref: 77 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292486:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995392 00000100:00100000:2.0:1785220582.292489:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995392 00010000:00010000:2.0:1785220582.292502:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000447f9a45/0xf62499ab92793500 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9e5 expref: 78 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292521:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995776 00000100:00100000:2.0:1785220582.292524:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995776 00010000:00010000:2.0:1785220582.292536:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dae04e73/0xf62499ab927932d7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7bc expref: 106 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292553:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996032 00000100:00100000:2.0:1785220582.292556:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996032 00010000:00010000:2.0:1785220582.292568:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000571fa8e8/0xf62499ab927934f2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d7 expref: 79 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292583:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995904 00000100:00100000:2.0:1785220582.292586:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995904 00010000:00010000:2.0:1785220582.292599:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c02efd77/0xf62499ab927934e4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c9 expref: 80 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.292628:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100687 00000100:00100000:2.0:1785220582.292636:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2404:x1871938519995008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.292650:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.292656:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000b42445f/0xf62499ab927932ad lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f792 expref: 106 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292667:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000b42445f/0xf62499ab927932ad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f792 expref: 106 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292693:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.292698:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.292705:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292715:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292731:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.292778:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+106:2404:x1871938519995008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 142us (661us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.292786:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100687 00000100:00100000:2.0:1785220582.292831:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100688 00000100:00100000:2.0:1785220582.292838:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+105:2404:x1871938519995264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.292846:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.292850:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000014a11cd6/0xf62499ab927932c9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ae expref: 105 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292859:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000014a11cd6/0xf62499ab927932c9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ae expref: 105 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292871:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.292874:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.292878:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292886:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292897:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.292915:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+105:2404:x1871938519995264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (774us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.292921:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100688 00000100:00100000:2.0:1785220582.292935:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100689 00000100:00100000:2.0:1785220582.292941:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+80:2404:x1871938519995520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.292948:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.292953:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000110c9113/0xf62499ab927934d6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9bb expref: 80 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292963:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000110c9113/0xf62499ab927934d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9bb expref: 80 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292978:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7322:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.292981:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7322:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.292985:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.292992:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293008:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.293025:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+80:2404:x1871938519995520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (867us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.293031:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100689 00000100:00100000:2.0:1785220582.293044:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100690 00000100:00100000:2.0:1785220582.293051:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+79:2404:x1871938519995136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.293058:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.293063:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000071a5ad15/0xf62499ab927934c8 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ad expref: 79 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293072:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000071a5ad15/0xf62499ab927934c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ad expref: 79 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293083:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7321:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293104:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7321:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293109:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293116:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293128:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.293145:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+79:2404:x1871938519995136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (967us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.293155:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100690 00000100:00100000:2.0:1785220582.293168:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100691 00000100:00100000:2.0:1785220582.293174:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+78:2404:x1871938519995392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.293182:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.293187:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000447f9a45/0xf62499ab92793500 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9e5 expref: 78 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293196:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000447f9a45/0xf62499ab92793500 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9e5 expref: 78 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293207:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7325:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293210:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7325:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293213:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293220:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293231:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.293246:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+78:2404:x1871938519995392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 73us (1052us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.293251:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100691 00000100:00100000:2.0:1785220582.293263:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100692 00000100:00100000:2.0:1785220582.293269:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+104:2407:x1871938519995776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.293279:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.293284:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dae04e73/0xf62499ab927932d7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7bc expref: 104 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293293:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dae04e73/0xf62499ab927932d7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7bc expref: 104 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293305:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293309:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293312:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293319:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293329:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.293344:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+104:2407:x1871938519995776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (1084us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.293351:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100692 00000100:00100000:2.0:1785220582.293362:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100693 00000100:00100000:2.0:1785220582.293368:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+77:2407:x1871938519996032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.293375:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.293380:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000571fa8e8/0xf62499ab927934f2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d7 expref: 77 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293396:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000571fa8e8/0xf62499ab927934f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d7 expref: 77 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293408:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7324:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293412:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7324:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293416:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293424:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293434:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.293450:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+77:2407:x1871938519996032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 82us (1171us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.293456:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100693 00000100:00100000:2.0:1785220582.293468:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100694 00000100:00100000:2.0:1785220582.293474:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e11450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2407:x1871938519995904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.293481:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.293487:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c02efd77/0xf62499ab927934e4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c9 expref: 76 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293497:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c02efd77/0xf62499ab927934e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c9 expref: 76 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293513:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7323:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293516:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7323:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293521:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293529:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293541:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.293558:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e11450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2407:x1871938519995904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 84us (1259us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.293564:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100694 00000100:00100000:2.0:1785220582.293630:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.293647:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519995648 00000100:00100000:2.0:1785220582.293651:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519995648 00010000:00010000:2.0:1785220582.293665:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000082d294a6/0xf62499ab927934ba lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f99f expref: 76 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.293684:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100695 00000100:00100000:2.0:1785220582.293690:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2405:x1871938519995648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.293698:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.293702:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000082d294a6/0xf62499ab927934ba lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f99f expref: 76 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293712:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000082d294a6/0xf62499ab927934ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f99f expref: 76 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293724:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7320:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293728:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7320:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.293732:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293741:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.293766:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.293781:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2405:x1871938519995648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (151us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.293787:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100695 00010000:00010000:1.0:1785220582.294124:0:14201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b42445f/0xf62499ab927932ad lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f792 expref: 103 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.294140:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0001_UUID:14201:1871938510773888:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.294161:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294171:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294178:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294188:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294195:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294216:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.294367:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d955fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+1353:3654:x1871938510773376:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12915us (12965us total) trans 4295070204 rc 0/0 00000100:00100000:1.0:1785220582.294376:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284755 00010000:00010000:1.0:1785220582.294409:0:14172:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000110c9113/0xf62499ab927934d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9bb expref: 75 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.294422:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0000_UUID:14172:1871938510775552:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.294434:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294442:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294449:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294457:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294464:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294479:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.294563:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91478700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+158:3654:x1871938510774144:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12494us (12549us total) trans 4295070779 rc 0/0 00000100:00100000:1.0:1785220582.294570:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284757 00010000:00010000:3.0:1785220582.294583:0:8440:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000014a11cd6/0xf62499ab927932c9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ae expref: 102 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.294595:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8440:1871938510774784:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.294597:0:14159:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082d294a6/0xf62499ab927934ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f99f expref: 74 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294609:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294618:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.294619:0:14159:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d998b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_021:lustre-OST0000_UUID:14159:1871938510773248:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.294625:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294630:0:14159:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294638:0:14159:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294639:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294646:0:14159:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294648:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294654:0:14159:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294661:0:14159:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294662:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220582.294672:0:14159:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:1.0:1785220582.294736:0:14159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_021:lustre-MDT0000-mdtlov_UUID+158:3654:x1871938510773120:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14747us (15251us total) trans 4295070780 rc 0/0 00000100:00100000:3.0:1785220582.294739:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+1353:3652:x1871938510773632:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12310us (12673us total) trans 4295070205 rc 0/0 00000100:00100000:1.0:1785220582.294743:0:14159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284754 00000100:00100000:3.0:1785220582.294746:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284758 00010000:00010000:3.0:1785220582.294775:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.294775:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:3.0:1785220582.294781:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294781:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294790:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.294791:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.294795:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294795:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294806:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294806:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294820:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294827:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009bf29279/0xf62499ab927a592b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294830:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294837:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a595c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294847:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.294851:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.294851:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294854:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294858:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.294861:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.294861:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294864:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294868:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294871:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294878:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294880:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294885:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000009e8b804/0xf62499ab927a5939 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294888:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5924 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.294897:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.294899:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.294900:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.294900:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.294902:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.294903:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.294907:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.294925:0:14267:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000447f9a45/0xf62499ab92793500 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9e5 expref: 73 pid: 13867 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.294936:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0000_UUID:14267:1871938510775424:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.294950:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294958:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294965:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294974:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.294982:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295000:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.295028:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.295033:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.295041:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.295044:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.295052:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.295062:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.295075:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c028e57/0xf62499ab927a5955 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.295076:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c928a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+158:3654:x1871938510774656:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12226us (12593us total) trans 4295070781 rc 0/0 00000100:00100000:3.0:1785220582.295082:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284762 00010000:00010000:2.0:1785220582.295107:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.295109:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.295111:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.295223:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510776832:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.295236:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.295300:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.295436:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a9bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510776960:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.295472:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.295532:0:14267:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510776960 00000100:00100000:3.0:1785220582.295539:0:14267:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510776960 00000100:00100000:3.0:1785220582.295578:0:14267:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284769 00010000:00010000:0.0:1785220582.295580:0:14223:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a5ad15/0xf62499ab927934c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ad expref: 72 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.295586:0:14267:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1353:3653:x1871938510776960:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.295593:0:14223:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0000_UUID:14223:1871938510774016:192.168.201.51@tcp:104: 00002000:00080000:3.0:1785220582.295607:0:14267:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29441 count 1 00010000:00010000:0.0:1785220582.295616:0:14223:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.295624:0:14223:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295625:0:14267:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.295632:0:14223:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.295641:0:14223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295645:0:14267:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d8fe9e4c/0xf62499ab9279330f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7f4 expref: 101 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295653:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7301:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.295654:0:14223:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295657:0:14267:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7301:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.295662:0:14267:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.295667:0:14223:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.295684:0:14267:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d8fe9e4c/0xf62499ab9279330f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7f4 expref: 101 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295696:0:14267:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d8fe9e4c/0xf62499ab9279330f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7f4 expref: 101 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.295707:0:14267:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510777088:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.295762:0:14267:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a5a9d80 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.295772:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.295776:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295783:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.295786:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295798:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295809:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295815:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d6b485cc/0xf62499ab927a5932 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295830:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.295831:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.295833:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.295853:0:14190:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000571fa8e8/0xf62499ab927934f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d7 expref: 71 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.295864:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0000_UUID:14190:1871938510775168:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.295877:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295886:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295893:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295905:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295912:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.295924:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.295966:0:14223:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+159:3654:x1871938510773760:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14237us (14273us total) trans 4295070782 rc 0/0 00000100:00100000:0.0:1785220582.295973:0:14223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284756 00000100:00100000:3.0:1785220582.295994:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c928700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+158:3654:x1871938510774528:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13277us (13593us total) trans 4295070783 rc 0/0 00000100:00100000:3.0:1785220582.296000:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284761 00010000:00010000:0.0:1785220582.296002:0:14210:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dae04e73/0xf62499ab927932d7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7bc expref: 101 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.296012:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510774912:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.296016:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.296021:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296023:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.296029:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.296031:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.296032:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296039:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.296041:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296047:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.296052:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296054:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.296060:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a594e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.296075:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.296077:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1785220582.296078:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.296111:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4ca00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.296115:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.296170:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+1354:3652:x1871938510774272:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13643us (13934us total) trans 4295070206 rc 0/0 00000100:00100000:0.0:1785220582.296177:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284759 00010000:00010000:0.0:1785220582.296208:0:14168:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c02efd77/0xf62499ab927934e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c9 expref: 70 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.296218:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0000_UUID:14168:1871938510775040:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.296229:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296236:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296242:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296249:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296256:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296269:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.296338:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+158:3652:x1871938510774400:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13689us (14017us total) trans 4295070784 rc 0/0 00000100:00100000:0.0:1785220582.296344:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284760 00000100:00100000:0.0:1785220582.296372:0:14191:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_048:lustre-OST0001_UUID:14191:1871938510775936:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.296385:0:14191:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.296410:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0000_UUID:14205:1871938510776064:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.296421:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.296446:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510776448:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.296455:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.296476:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510776576:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.296487:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.296509:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510776320:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.296519:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296530:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.296534:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296541:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4dc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.296544:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296550:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296561:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296567:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000faa65968/0xf62499ab927a5947 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.296581:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.296583:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.296585:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.296675:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c928380/2, rc 0, transno 4295070781 00000100:00100000:0.0:1785220582.296688:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c928380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510774656:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.296733:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8c92a680/2, rc 0, transno 4295070783 00000100:00100000:0.0:1785220582.296741:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c92a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510774528:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.296783:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d91479880/2, rc 0, transno 4295070779 00000100:00100000:0.0:1785220582.296790:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91479880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510774144:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.296820:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fd180/2, rc 0, transno 4295070782 00000100:00100000:0.0:1785220582.296827:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510773760:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.296855:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d915aca80/2, rc 0, transno 4295070204 00000100:00100000:0.0:1785220582.296864:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d915aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510773376:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.296894:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b008380/2, rc 0, transno 4295070780 00000100:00100000:0.0:1785220582.296900:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510773120:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.296926:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a98000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777216:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.296953:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.296972:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a98e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777728:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.296988:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.297001:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297004:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1785220582.297009:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a98a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777344:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.297014:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297021:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f92f3013/0xf62499ab927a5940 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297026:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.297039:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.297040:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a9aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777856:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.297041:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.297043:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.297054:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510777216 00000100:00100000:0.0:1785220582.297057:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.297058:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510777216 00000100:00100000:0.0:1785220582.297082:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a99f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510777600:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.297122:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.297144:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510777728 00000100:00100000:0.0:1785220582.297148:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510777728 00000100:00100000:0.0:1785220582.297169:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510777344 00000100:00100000:0.0:1785220582.297172:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510777344 00000100:00100000:0.0:1785220582.297187:0:14192:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510777856 00000100:00100000:0.0:1785220582.297189:0:14192:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510777856 00000100:00100000:0.0:1785220582.297205:0:14192:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284770 00000100:00100000:0.0:1785220582.297212:0:14192:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97a9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+160:3654:x1871938510777216:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.297230:0:14192:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29480 count 1 00010000:00010000:0.0:1785220582.297246:0:14192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.297258:0:14192:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000010a8adf1/0xf62499ab9279352a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa0f expref: 69 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.297266:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7328:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.297270:0:14192:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7328:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.297274:0:14192:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.297294:0:14192:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000010a8adf1/0xf62499ab9279352a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa0f expref: 69 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.297305:0:14192:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000010a8adf1/0xf62499ab9279352a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa0f expref: 69 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297306:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510777600 00000100:00100000:3.0:1785220582.297309:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510777600 00000100:00100000:0.0:1785220582.297317:0:14192:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510777984:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.297335:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284771 00000100:00100000:3.0:1785220582.297340:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97a9b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1354:3654:x1871938510777728:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.297349:0:14192:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00002000:00080000:3.0:1785220582.297353:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29442 count 1 00000100:00100000:0.0:1785220582.297364:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:3.0:1785220582.297366:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297369:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97a99880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510777472:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.297377:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d1c110b6/0xf62499ab9279331d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f802 expref: 100 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297384:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7302:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.297388:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7302:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.297390:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.297392:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297409:0:14197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510777472 00010000:00010000:3.0:1785220582.297410:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d1c110b6/0xf62499ab9279331d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f802 expref: 100 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297411:0:14197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510777472 00010000:00010000:3.0:1785220582.297420:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d1c110b6/0xf62499ab9279331d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f802 expref: 100 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297429:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510778112:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.297430:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284772 00000100:00100000:0.0:1785220582.297436:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97a99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+162:3654:x1871938510777344:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.297446:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29481 count 1 00000100:00100000:3.0:1785220582.297452:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920d80 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.297458:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297466:0:14186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284773 00010000:00010000:0.0:1785220582.297469:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000073fdf4cc/0xf62499ab92793538 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa1d expref: 69 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297473:0:14186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97a9b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+162:3654:x1871938510777856:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.297476:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7329:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.297479:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7329:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.297482:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1785220582.297483:0:14186:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29484 count 1 00010000:00010000:3.0:1785220582.297492:0:14186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.297498:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000073fdf4cc/0xf62499ab92793538 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa1d expref: 69 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297509:0:14186:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000734027a9/0xf62499ab92793562 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa47 expref: 69 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.297509:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000073fdf4cc/0xf62499ab92793538 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa1d expref: 69 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297517:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x732c:0x0].0x0 since=17s v=(0 0) 00000100:00100000:0.0:1785220582.297519:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510778240:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.297520:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.297524:0:14186:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297539:0:14186:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000734027a9/0xf62499ab92793562 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa47 expref: 69 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297539:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.297548:0:14186:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000734027a9/0xf62499ab92793562 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa47 expref: 69 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297555:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284774 00000100:00100000:3.0:1785220582.297558:0:14186:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0000_UUID:14186:1871938510778368:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.297561:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+162:3653:x1871938510777600:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.297571:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29483 count 1 00000100:00100000:3.0:1785220582.297579:0:14186:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a920300 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.297582:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297593:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284775 00010000:00010000:0.0:1785220582.297593:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f71846aa/0xf62499ab92793554 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa39 expref: 69 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297600:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+162:3653:x1871938510777472:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.297600:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x732b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.297603:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732b:0x0].0x0 since=0s v=(1 0) 00002000:00080000:3.0:1785220582.297610:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29482 count 1 00010000:00010000:3.0:1785220582.297620:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297631:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000074987f5f/0xf62499ab92793546 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa2b expref: 69 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297643:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x732a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.297646:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.297649:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297667:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000074987f5f/0xf62499ab92793546 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa2b expref: 69 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.297677:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000074987f5f/0xf62499ab92793546 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa2b expref: 69 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.297684:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297686:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b032300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510778496:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.297704:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f71846aa/0xf62499ab92793554 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa39 expref: 69 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.297708:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c600 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.297714:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f71846aa/0xf62499ab92793554 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa39 expref: 69 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.297723:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510778624:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.297743:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.297796:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.297828:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.297852:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.297871:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.297887:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.297909:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:0.0:1785220582.297940:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9147b480/2, rc 0, transno 4295070784 00000100:00100000:0.0:1785220582.297948:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9147b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510774400:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.297977:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.297985:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510778752:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.298007:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.298027:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510778752 00000100:00100000:0.0:1785220582.298030:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510778752 00000100:00100000:0.0:1785220582.298050:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284776 00000100:00100000:0.0:1785220582.298056:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+163:3653:x1871938510778752:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.298067:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29485 count 1 00010000:00010000:0.0:1785220582.298078:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.298108:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003ccfdb7e/0xf62499ab92793570 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa55 expref: 69 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.298115:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x732d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.298118:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.298122:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 4/0,1 mode: --/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.298140:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003ccfdb7e/0xf62499ab92793570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa55 expref: 69 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.298149:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003ccfdb7e/0xf62499ab92793570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa55 expref: 69 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.298159:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510778880:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.298182:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.298205:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9147b100/2, rc 0, transno 4295070206 00000100:00100000:0.0:1785220582.298214:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9147b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510774272:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.298245:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510779008:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.298268:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.298300:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00a680/2, rc 0, transno 4295070205 00000100:00100000:0.0:1785220582.298307:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510773632:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.298335:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510779136:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.298357:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.298457:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.298460:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996160 00000100:00100000:3.0:1785220582.298464:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996160 00000100:00100000:1.0:1785220582.298467:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996416 00000100:00100000:1.0:1785220582.298470:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996416 00010000:00010000:3.0:1785220582.298487:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000821f5aeb/0xf62499ab927932bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a0 expref: 102 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.298487:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002524ecf5/0xf62499ab927932f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d8 expref: 102 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.298514:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996672 00000100:00100000:3.0:1785220582.298517:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996672 00000100:00100000:1.0:1785220582.298519:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996288 00000100:00100000:1.0:1785220582.298521:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996288 00010000:00010000:3.0:1785220582.298529:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002ea63c77/0xf62499ab9279351c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa01 expref: 71 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.298530:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a1877633/0xf62499ab9279350e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9f3 expref: 71 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.298546:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996544 00000100:00100000:3.0:1785220582.298549:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996544 00010000:00010000:3.0:1785220582.298562:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ffdf4c5b/0xf62499ab927932e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ca expref: 103 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.298809:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996800 00000100:00100000:1.0:1785220582.298813:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996800 00000100:00100000:3.0:1785220582.298815:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100696 00010000:00010000:1.0:1785220582.298827:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002df7809b/0xf62499ab92793301 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7e6 expref: 104 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.298833:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+104:2406:x1871938519996160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.298843:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.298848:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100697 00010000:00010000:3.0:1785220582.298849:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000821f5aeb/0xf62499ab927932bb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a0 expref: 104 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.298854:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+104:2406:x1871938519996416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.298860:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000821f5aeb/0xf62499ab927932bb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a0 expref: 104 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.298861:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.298865:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002524ecf5/0xf62499ab927932f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d8 expref: 104 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.298872:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002524ecf5/0xf62499ab927932f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d8 expref: 104 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.298877:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000821f5aeb/0xf62499ab927932bb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7a0 expref: 104 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.298883:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002524ecf5/0xf62499ab927932f3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7d8 expref: 104 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.298890:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.298892:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.298893:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.298895:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72ff:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.298899:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.298900:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.298910:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.298910:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.298930:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.298930:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.298953:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+102:2406:x1871938519996160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 121us (1157us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.298963:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100696 00000100:00100000:1.0:1785220582.298979:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+101:2406:x1871938519996416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 125us (1150us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.299010:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100698 00000100:00100000:0.0:1785220582.299018:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+71:2406:x1871938519996672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.299026:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.299031:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002ea63c77/0xf62499ab9279351c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa01 expref: 71 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299040:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002ea63c77/0xf62499ab9279351c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa01 expref: 71 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299053:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ea63c77/0xf62499ab9279351c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa01 expref: 71 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299062:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7327:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.299065:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7327:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.299069:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299076:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299115:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.299131:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+70:2406:x1871938519996672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 115us (1278us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.299138:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100698 00000100:00100000:0.0:1785220582.299152:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100699 00000100:00100000:0.0:1785220582.299158:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+69:2406:x1871938519996288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.299166:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.299171:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a1877633/0xf62499ab9279350e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9f3 expref: 69 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299180:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a1877633/0xf62499ab9279350e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9f3 expref: 69 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299192:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1877633/0xf62499ab9279350e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9f3 expref: 69 pid: 14209 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299201:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7326:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.299204:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7326:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.299208:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299216:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299238:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.299253:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+68:2406:x1871938519996288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1381us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.299260:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100699 00000100:00100000:0.0:1785220582.299275:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100700 00000100:00100000:0.0:1785220582.299281:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+101:2406:x1871938519996544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.299290:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.299294:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ffdf4c5b/0xf62499ab927932e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ca expref: 101 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299304:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ffdf4c5b/0xf62499ab927932e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ca expref: 101 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299316:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ffdf4c5b/0xf62499ab927932e5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7ca expref: 101 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299325:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.299328:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x72fe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.299332:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299338:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299360:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.299375:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+100:2406:x1871938519996544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (1486us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.299381:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100700 00000100:00100000:3.0:1785220582.299386:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100701 00000100:00100000:3.0:1785220582.299394:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce7a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+99:2406:x1871938519996800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.299401:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299402:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.299407:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002df7809b/0xf62499ab92793301 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7e6 expref: 99 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299411:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299417:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002df7809b/0xf62499ab92793301 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7e6 expref: 99 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299421:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.299429:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299430:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002df7809b/0xf62499ab92793301 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7e6 expref: 99 pid: 14171 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299440:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7300:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.299443:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7300:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.299445:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.299446:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299453:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299469:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.299483:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce7a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+98:2406:x1871938519996800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (1574us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.299490:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100701 00010000:00010000:3.0:1785220582.299512:0:14191:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299519:0:14191:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299526:0:14191:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299533:0:14191:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.299544:0:14191:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.299594:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+164:3654:x1871938510776192:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13506us (14150us total) trans 4295070785 rc 0/0 00000100:00100000:0.0:1785220582.299601:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284767 00000100:00100000:3.0:1785220582.299630:0:14191:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_048:lustre-MDT0000-mdtlov_UUID+1355:3653:x1871938510773504:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14952us (15176us total) trans 4295070207 rc 0/0 00000100:00100000:3.0:1785220582.299637:0:14191:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284763 00000100:00100000:0.0:1785220582.299759:0:14250:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510779008 00000100:00100000:0.0:1785220582.299765:0:14250:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510779008 00000100:00100000:1.0:1785220582.299825:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100697 00010000:00010000:1.0:1785220582.299867:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.299876:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.299884:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.299891:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.299905:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220582.299984:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1357:3653:x1871938510775680:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14449us (14677us total) trans 4295070208 rc 0/0 00000100:00100000:1.0:1785220582.299991:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284765 00000100:00100000:2.0:1785220582.300034:0:13820:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510779136 00000100:00100000:2.0:1785220582.300038:0:13820:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510779136 00000100:00100000:0.0:1785220582.300128:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284777 00000100:00100000:0.0:1785220582.300136:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1357:3653:x1871938510779008:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.300155:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29443 count 1 00010000:00010000:0.0:1785220582.300171:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300184:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000115d3ce7/0xf62499ab9279332b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f810 expref: 96 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300191:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7303:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.300194:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7303:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.300198:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300224:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000115d3ce7/0xf62499ab9279332b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f810 expref: 96 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300237:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000115d3ce7/0xf62499ab9279332b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f810 expref: 96 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.300250:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510779264:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.300290:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.300307:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300317:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300325:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300333:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300347:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220582.300426:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+164:3654:x1871938510775808:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15376us (15814us total) trans 4295070786 rc 0/0 00000100:00100000:0.0:1785220582.300433:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284764 00010000:00010000:0.0:1785220582.300453:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300462:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300470:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300477:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300486:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00000100:00100000:0.0:1785220582.300565:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+1358:3652:x1871938510775296:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14895us (15210us total) trans 4295070209 rc 0/0 00000100:00100000:0.0:1785220582.300570:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284766 00010000:00010000:0.0:1785220582.300588:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:0.0:1785220582.300595:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300603:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.300618:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300629:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300643:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300649:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a597f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300666:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220582.300670:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300677:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4de00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.300680:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300688:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300698:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300705:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f648d795/0xf62499ab927a5963 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300719:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220582.300722:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300728:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4c200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.300730:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300737:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300746:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300764:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000820fd93c/0xf62499ab927a5971 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72ff:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300777:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.300780:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300786:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ff4d200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.300789:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300796:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300805:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300813:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000072eeceb1/0xf62499ab927a596a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300827:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.300830:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300836:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.300838:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300844:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300853:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300860:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5978 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x72fe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.300872:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.300874:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.300876:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.300883:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.300916:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93609180/2, rc 0, transno 4295070785 00000100:00100000:0.0:1785220582.300928:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510776192:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.300978:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a4e00/2, rc 0, transno 4295070786 00000100:00100000:0.0:1785220582.300986:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510775808:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.300992:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.301018:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510779392:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.301045:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.301071:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.301103:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a6d80/2, rc 0, transno 4295070208 00000100:00100000:3.0:1785220582.301110:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510779392 00000100:00100000:0.0:1785220582.301112:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510775680:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.301114:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510779392 00000004:00080000:0.0:1785220582.301148:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d311880/2, rc 0, transno 4295070207 00000100:00100000:3.0:1785220582.301152:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284778 00000100:00100000:0.0:1785220582.301154:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510773504:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.301159:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+164:3654:x1871938510779392:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.301173:0:14259:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29486 count 1 00000100:00100000:0.0:1785220582.301183:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510779776:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.301188:0:14259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.301201:0:14259:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab9abaa4/0xf62499ab9279357e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa63 expref: 67 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.301203:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.301214:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x732e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.301218:0:14259:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.301222:0:14259:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 4/0,1 mode: --/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.301227:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94aa2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510779648:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.301244:0:14259:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ab9abaa4/0xf62499ab9279357e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa63 expref: 67 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.301246:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.301254:0:14259:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ab9abaa4/0xf62499ab9279357e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa63 expref: 67 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.301261:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1785220582.301265:0:14259:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510779904:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.301265:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510779520:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.301286:0:14259:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c240 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.301301:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510779776 00000100:00100000:3.0:1785220582.301304:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510779776 00000100:00100000:3.0:1785220582.301326:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510779648 00000100:00100000:3.0:1785220582.301329:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510779648 00000100:00100000:3.0:1785220582.301345:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284779 00000100:00100000:3.0:1785220582.301351:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94aa0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1359:3654:x1871938510779776:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.301362:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29446 count 1 00010000:00010000:3.0:1785220582.301375:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.301387:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009927c5ac/0xf62499ab92793355 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f83a expref: 96 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.301398:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7306:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.301401:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7306:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.301405:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.301411:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284780 00000100:00100000:1.0:1785220582.301420:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97984700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1358:3653:x1871938510779648:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.301422:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009927c5ac/0xf62499ab92793355 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f83a expref: 96 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.301430:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29445 count 1 00010000:00010000:3.0:1785220582.301432:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009927c5ac/0xf62499ab92793355 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f83a expref: 96 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.301441:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.301442:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b032d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510780032:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.301451:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007ff7ad65/0xf62499ab92793347 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f82c expref: 96 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.301458:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7305:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.301465:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7305:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.301470:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.301489:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007ff7ad65/0xf62499ab92793347 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f82c expref: 96 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.301499:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007ff7ad65/0xf62499ab92793347 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f82c expref: 96 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.301509:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510780160:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.301552:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.301588:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a7b80/2, rc 0, transno 4295070209 00000100:00100000:0.0:1785220582.301597:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510775296:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.301643:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.301649:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97985180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510780288:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.301669:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.301690:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510779520 00000100:00100000:0.0:1785220582.301694:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510779520 00000100:00100000:0.0:1785220582.301715:0:14262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510780288 00000100:00100000:0.0:1785220582.301718:0:14262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510780288 00000100:00100000:0.0:1785220582.301736:0:14262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284781 00000100:00100000:0.0:1785220582.301742:0:14262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97985f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+165:3653:x1871938510779520:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.301764:0:14262:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29487 count 1 00010000:00010000:0.0:1785220582.301777:0:14262:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301790:0:14262:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002abfd280/0xf62499ab9279358c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa71 expref: 67 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301796:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x732f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.301798:0:14262:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.301802:0:14262:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301820:0:14262:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002abfd280/0xf62499ab9279358c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa71 expref: 67 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301830:0:14262:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002abfd280/0xf62499ab9279358c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa71 expref: 67 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.301845:0:14262:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97985c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510780416:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.301871:0:14262:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.301886:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284782 00000100:00100000:0.0:1785220582.301893:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97985880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1359:3654:x1871938510780288:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.301904:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29447 count 1 00010000:00010000:0.0:1785220582.301915:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301926:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000526b0daf/0xf62499ab92793363 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f848 expref: 96 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301933:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7307:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.301936:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7307:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.301939:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301955:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000526b0daf/0xf62499ab92793363 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f848 expref: 96 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.301964:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000526b0daf/0xf62499ab92793363 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f848 expref: 96 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.301973:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97987480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510780544:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.301993:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.302179:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.302269:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284783 00000100:00100000:2.0:1785220582.302279:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1359:3654:x1871938510779136:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.302292:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29444 count 1 00010000:00010000:2.0:1785220582.302304:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302315:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007e380aca/0xf62499ab92793339 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f81e expref: 96 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302321:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7304:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.302324:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7304:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.302328:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302351:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007e380aca/0xf62499ab92793339 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f81e expref: 96 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302361:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007e380aca/0xf62499ab92793339 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f81e expref: 96 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.302370:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510780672:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.302393:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bae40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.302416:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302424:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302432:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302439:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.302454:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.302581:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9573d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+1360:3653:x1871938510776704:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16042us (16172us total) trans 4295070210 rc 0/0 00000100:00100000:2.0:1785220582.302588:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284768 00000004:00080000:0.0:1785220582.302714:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a98380/2, rc 0, transno 4295070210 00000100:00100000:0.0:1785220582.302723:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a98380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510776704:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.302769:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.302777:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97986300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510780800:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.302801:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.302824:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510780800 00000100:00100000:0.0:1785220582.302828:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510780800 00000100:00100000:0.0:1785220582.302852:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284784 00000100:00100000:0.0:1785220582.302858:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97984e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1360:3653:x1871938510780800:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.302869:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29448 count 1 00010000:00010000:0.0:1785220582.302881:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.302893:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007c46a318/0xf62499ab92793371 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f856 expref: 96 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.302899:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7308:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.302901:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7308:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.302905:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.302925:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007c46a318/0xf62499ab92793371 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f856 expref: 96 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.302934:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007c46a318/0xf62499ab92793371 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f856 expref: 96 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.302944:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510780928:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.302964:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.303123:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.303129:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.303138:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.303142:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.303151:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.303168:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.303176:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5986 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.303205:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.303208:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.303210:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.303878:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c300 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.303925:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.304054:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.304143:0:14267:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_087:lustre-OST0001_UUID:14267:1871938510777088:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.304165:0:14267:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.304332:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519996928 00000100:00100000:1.0:1785220582.304337:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519996928 00010000:00010000:1.0:1785220582.304364:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d8fe9e4c/0xf62499ab9279330f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7f4 expref: 97 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.304403:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100702 00000100:00100000:1.0:1785220582.304412:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+97:2407:x1871938519996928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.304423:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.304429:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d8fe9e4c/0xf62499ab9279330f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7f4 expref: 97 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.304440:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d8fe9e4c/0xf62499ab9279330f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7f4 expref: 97 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.304458:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8fe9e4c/0xf62499ab9279330f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f7f4 expref: 97 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.304471:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7301:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.304475:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7301:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.304480:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.304489:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.304516:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.304585:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+96:2407:x1871938519996928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 173us (531us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.304593:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100702 00010000:00010000:3.0:1785220582.304601:0:14267:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.304610:0:14267:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.304625:0:14267:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.304631:0:14267:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.304654:0:14267:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.304690:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.304697:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.304705:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.304708:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.304715:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.304728:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.304734:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a598d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7301:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.304760:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.304762:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.304764:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.304809:0:14267:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_087:lustre-MDT0000-mdtlov_UUID+1361:3653:x1871938510776960:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9223us (9335us total) trans 4295070211 rc 0/0 00000100:00100000:3.0:1785220582.304816:0:14267:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284769 00000004:00080000:0.0:1785220582.304841:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a9bb80/2, rc 0, transno 4295070211 00000100:00100000:0.0:1785220582.304860:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a9bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510776960:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.304991:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.305003:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfc4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510781056:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.305037:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.305150:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510781056 00000100:00100000:0.0:1785220582.305155:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510781056 00000100:00100000:0.0:1785220582.305183:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284785 00000100:00100000:0.0:1785220582.305190:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1361:3652:x1871938510781056:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.305209:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29449 count 1 00010000:00010000:0.0:1785220582.305226:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.305239:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd1b5920/0xf62499ab9279337f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f864 expref: 95 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.305247:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7309:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.305250:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7309:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.305255:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.305276:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fd1b5920/0xf62499ab9279337f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f864 expref: 95 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.305288:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fd1b5920/0xf62499ab9279337f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f864 expref: 95 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.305299:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510781184:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.305333:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.307262:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.307362:0:14192:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_049:lustre-OST0000_UUID:14192:1871938510777984:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.307392:0:14192:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.308882:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0000_UUID:14197:1871938510778240:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.308910:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.309079:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510778112:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.309119:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.309275:0:13817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997056 00000100:00100000:1.0:1785220582.309282:0:13817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997056 00010000:00010000:1.0:1785220582.309317:0:13817:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000010a8adf1/0xf62499ab9279352a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa0f expref: 68 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.309390:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100703 00000100:00100000:1.0:1785220582.309400:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90e8e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+68:2407:x1871938519997056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.309415:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.309421:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000010a8adf1/0xf62499ab9279352a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa0f expref: 68 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.309433:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000010a8adf1/0xf62499ab9279352a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa0f expref: 68 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.309456:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010a8adf1/0xf62499ab9279352a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa0f expref: 68 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.309470:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7328:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.309475:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7328:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.309481:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.309491:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.309523:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.309573:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90e8e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+67:2407:x1871938519997056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 174us (2319us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.309581:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100703 00010000:00010000:2.0:1785220582.309636:0:14192:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309650:0:14192:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309660:0:14192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309667:0:14192:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309683:0:14192:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.309773:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.309856:0:14192:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97a9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_049:lustre-MDT0000-mdtlov_UUID+166:3654:x1871938510777216:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12644us (12901us total) trans 4295070787 rc 0/0 00000100:00100000:2.0:1785220582.309865:0:14192:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284770 00010000:00010000:2.0:1785220582.309890:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.309896:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309904:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.309908:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309917:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309928:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309935:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a1704ba/0xf62499ab927a5994 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.309954:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.309956:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.309958:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.311593:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997312 00000100:00100000:0.0:1785220582.311605:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997312 00010000:00010000:0.0:1785220582.311651:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000073fdf4cc/0xf62499ab92793538 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa1d expref: 67 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.311681:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100704 00000100:00100000:0.0:1785220582.311760:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b033100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+67:2407:x1871938519997312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.311778:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.311784:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000073fdf4cc/0xf62499ab92793538 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa1d expref: 67 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.311795:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000073fdf4cc/0xf62499ab92793538 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa1d expref: 67 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.311815:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073fdf4cc/0xf62499ab92793538 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa1d expref: 67 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.311830:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7329:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.311835:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7329:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.311840:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.311849:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.311886:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.311899:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.311908:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.311917:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.311924:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.311929:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b033100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+66:2407:x1871938519997312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 240us (2156us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.311936:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100704 00010000:00010000:1.0:1785220582.311939:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.311985:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0000_UUID:14203:1871938510778624:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.312005:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.312030:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0000_UUID:14165:1871938510778880:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.312040:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312053:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.312058:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312065:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.312068:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312076:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312121:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312130:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e343f9d2/0xf62499ab927a59a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312147:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.312148:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.312151:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.312179:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a98a80/2, rc 0, transno 4295070788 00000100:00100000:0.0:1785220582.312191:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a98a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777344:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.312250:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a98000/2, rc 0, transno 4295070787 00000100:00100000:0.0:1785220582.312257:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a98000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777216:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.312291:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510781312:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.312302:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.312322:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.312346:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510781312 00000100:00100000:0.0:1785220582.312350:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510781312 00000100:00100000:0.0:1785220582.312376:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284786 00000100:00100000:0.0:1785220582.312383:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90e8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+167:3654:x1871938510781312:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.312403:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29488 count 1 00000100:00100000:3.0:1785220582.312414:0:14186:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0000_UUID:14186:1871938510778368:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.312421:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.312429:0:14186:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312434:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000923c2c8/0xf62499ab9279359a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa7f expref: 65 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312440:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7330:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.312444:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7330:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.312449:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.312469:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b032300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510778496:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.312469:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000923c2c8/0xf62499ab9279359a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa7f expref: 65 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.312479:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.312481:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000923c2c8/0xf62499ab9279359a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa7f expref: 65 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.312494:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510781568:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.312533:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97a99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+166:3654:x1871938510777344:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14647us (15055us total) trans 4295070788 rc 0/0 00000100:00100000:0.0:1785220582.312534:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.312542:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284772 00000100:00100000:0.0:1785220582.312551:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.312557:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510781440:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.312561:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997184 00000100:00100000:1.0:1785220582.312564:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997184 00010000:00010000:1.0:1785220582.312580:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d1c110b6/0xf62499ab9279331d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f802 expref: 96 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.312607:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100705 00000100:00100000:1.0:1785220582.312614:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+96:2407:x1871938519997184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.312622:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.312627:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d1c110b6/0xf62499ab9279331d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f802 expref: 96 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.312636:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d1c110b6/0xf62499ab9279331d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f802 expref: 96 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.312650:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d1c110b6/0xf62499ab9279331d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f802 expref: 96 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.312661:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7302:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.312664:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7302:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.312668:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.312675:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.312694:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.314830:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0001_UUID:14250:1871938510779264:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.314837:0:14259:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_082:lustre-OST0000_UUID:14259:1871938510779904:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.314857:0:14259:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.314857:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.314896:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.314932:0:14262:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97985c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_085:lustre-OST0000_UUID:14262:1871938510780416:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.314945:0:14262:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.314969:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8442:1871938510780544:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.314980:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.314983:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b032d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510780032:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.314993:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315001:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510780928:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.315011:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315032:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0001_UUID:14198:1871938510781184:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.315033:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.315042:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315050:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.315069:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997568 00000100:00100000:3.0:1785220582.315073:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.315075:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997568 00010000:00010000:0.0:1785220582.315116:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003ccfdb7e/0xf62499ab92793570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa55 expref: 66 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315143:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997824 00000100:00100000:0.0:1785220582.315146:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997824 00010000:00010000:0.0:1785220582.315159:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000074987f5f/0xf62499ab92793546 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa2b expref: 67 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315179:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998080 00000100:00100000:0.0:1785220582.315182:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998080 00010000:00010000:0.0:1785220582.315193:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000115d3ce7/0xf62499ab9279332b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f810 expref: 96 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315215:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100706 00000100:00100000:0.0:1785220582.315224:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+67:2407:x1871938519997568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.315238:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.315244:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003ccfdb7e/0xf62499ab92793570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa55 expref: 67 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315255:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.315255:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003ccfdb7e/0xf62499ab92793570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa55 expref: 67 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315276:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ccfdb7e/0xf62499ab92793570 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa55 expref: 67 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315281:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.315292:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732d:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.315294:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998336 00010000:00010000:0.0:1785220582.315296:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732d:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.315297:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998336 00010000:00010000:0.0:1785220582.315302:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315311:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315312:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002abfd280/0xf62499ab9279358c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa71 expref: 67 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315330:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998592 00000100:00100000:3.0:1785220582.315333:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998592 00010000:00010000:0.0:1785220582.315334:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.315343:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007e380aca/0xf62499ab92793339 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f81e expref: 97 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315358:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100707 00000100:00100000:3.0:1785220582.315364:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+67:2407:x1871938519997824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.315368:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+67:2407:x1871938519997568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 146us (335us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.315371:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220582.315375:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100706 00010000:00010000:3.0:1785220582.315376:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000074987f5f/0xf62499ab92793546 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa2b expref: 67 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315384:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000074987f5f/0xf62499ab92793546 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa2b expref: 66 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315388:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100708 00000100:00100000:0.0:1785220582.315394:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+97:2407:x1871938519998080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.315398:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074987f5f/0xf62499ab92793546 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa2b expref: 66 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315401:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.315406:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000115d3ce7/0xf62499ab9279332b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f810 expref: 97 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315408:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.315411:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.315416:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315416:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000115d3ce7/0xf62499ab9279332b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f810 expref: 97 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315425:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315429:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000115d3ce7/0xf62499ab9279332b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f810 expref: 97 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315438:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7303:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.315440:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.315442:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7303:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.315445:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315452:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315456:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+65:2407:x1871938519997824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (405us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.315463:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100707 00010000:00010000:0.0:1785220582.315467:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.315475:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100709 00000100:00100000:3.0:1785220582.315482:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b032300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+64:2407:x1871938519998336:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.315482:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+96:2407:x1871938519998080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.315488:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100708 00010000:00010000:3.0:1785220582.315492:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.315496:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002abfd280/0xf62499ab9279358c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa71 expref: 64 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315498:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100710 00000100:00100000:0.0:1785220582.315503:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+95:2407:x1871938519998592:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.315506:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002abfd280/0xf62499ab9279358c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa71 expref: 64 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315512:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.315516:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007e380aca/0xf62499ab92793339 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f81e expref: 95 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315518:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002abfd280/0xf62499ab9279358c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa71 expref: 64 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315528:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.315531:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.315531:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007e380aca/0xf62499ab92793339 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f81e expref: 95 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315536:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315542:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7304:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.315543:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315545:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7304:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.315549:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315556:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315557:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.315566:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.315573:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b032300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+63:2407:x1871938519998336:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.315579:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+95:2407:x1871938519998592:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.315580:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100709 00000100:00100000:0.0:1785220582.315584:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100710 00010000:00010000:3.0:1785220582.315606:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315615:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315626:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315633:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315649:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.315833:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+167:3653:x1871938510777472:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18232us (18440us total) trans 4295070789 rc 0/0 00010000:00010000:0.0:1785220582.315835:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315841:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284775 00010000:00010000:0.0:1785220582.315845:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315853:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.315854:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510781440 00000100:00100000:3.0:1785220582.315857:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510781440 00010000:00010000:0.0:1785220582.315860:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315873:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.315877:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284787 00000100:00100000:3.0:1785220582.315882:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90e8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+167:3654:x1871938510781440:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.315901:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29489 count 1 00010000:00010000:3.0:1785220582.315920:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315932:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000045cbe019/0xf62499ab927935a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa8d expref: 62 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315940:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7331:0x0].0x0 since=17s v=(0 0) 00000100:00100000:0.0:1785220582.315941:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+168:3653:x1871938510778752:12345-0@lo:6:osp-syn-0-0.0 Request processed in 17886us (17933us total) trans 4295070790 rc 0/0 00010000:00010000:3.0:1785220582.315943:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7331:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.315947:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.315948:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284776 00010000:00010000:0.0:1785220582.315969:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.315975:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315977:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000045cbe019/0xf62499ab927935a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa8d expref: 62 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315984:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.315988:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.315989:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000045cbe019/0xf62499ab927935a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa8d expref: 62 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.315997:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316003:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510781696:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.316008:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316015:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a37a8de8/0xf62499ab927a59b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316030:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.316033:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316043:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c240 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.316044:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.316047:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316054:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316064:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 1/0,0 mode: --/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316071:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.316072:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d360efc/0xf62499ab927a59be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316225:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.316227:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.316228:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.316238:0:14262:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316246:0:14262:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316255:0:14262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316268:0:14262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316281:0:14262:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.316289:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.316308:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.316325:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.316343:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.316356:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998848 00000100:00100000:0.0:1785220582.316357:0:14262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97985f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_085:lustre-MDT0000-mdtlov_UUID+168:3653:x1871938510779520:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14614us (14804us total) trans 4295070791 rc 0/0 00000100:00100000:3.0:1785220582.316359:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998848 00000100:00100000:0.0:1785220582.316363:0:14262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284781 00010000:00010000:3.0:1785220582.316374:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009927c5ac/0xf62499ab92793355 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f83a expref: 95 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316381:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316389:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997440 00010000:00010000:0.0:1785220582.316389:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316391:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997440 00010000:00010000:0.0:1785220582.316397:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316403:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f71846aa/0xf62499ab92793554 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa39 expref: 63 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316404:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316417:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.316418:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997696 00000100:00100000:3.0:1785220582.316421:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997696 00010000:00010000:3.0:1785220582.316431:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000734027a9/0xf62499ab92793562 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa47 expref: 64 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316446:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519997952 00000100:00100000:3.0:1785220582.316448:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519997952 00010000:00010000:3.0:1785220582.316459:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007ff7ad65/0xf62499ab92793347 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f82c expref: 96 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316470:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998208 00000100:00100000:3.0:1785220582.316472:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998208 00010000:00010000:3.0:1785220582.316481:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ab9abaa4/0xf62499ab9279357e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa63 expref: 65 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316495:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100711 00000100:00100000:0.0:1785220582.316498:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+1362:3653:x1871938510779008:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16364us (18229us total) trans 4295070212 rc 0/0 00000100:00100000:3.0:1785220582.316500:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+96:2407:x1871938519998848:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.316504:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284777 00010000:00010000:3.0:1785220582.316508:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.316512:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009927c5ac/0xf62499ab92793355 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f83a expref: 96 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.316522:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100712 00010000:00010000:3.0:1785220582.316523:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009927c5ac/0xf62499ab92793355 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f83a expref: 96 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.316528:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b033b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+65:2407:x1871938519997440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.316535:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.316536:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009927c5ac/0xf62499ab92793355 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f83a expref: 96 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316540:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f71846aa/0xf62499ab92793554 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa39 expref: 65 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316547:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7306:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.316549:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f71846aa/0xf62499ab92793554 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa39 expref: 65 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316551:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7306:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.316555:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316560:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f71846aa/0xf62499ab92793554 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa39 expref: 65 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316563:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316570:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.316572:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.316576:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316579:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.316584:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.316597:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+95:2407:x1871938519998848:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 97us (526us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.316598:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.316603:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100711 00000100:00100000:3.0:1785220582.316617:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100713 00000100:00100000:3.0:1785220582.316624:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+64:2407:x1871938519997696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.316632:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.316637:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000734027a9/0xf62499ab92793562 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa47 expref: 64 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316646:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000734027a9/0xf62499ab92793562 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa47 expref: 64 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316659:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000734027a9/0xf62499ab92793562 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa47 expref: 64 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316668:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.316671:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.316675:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316683:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.316699:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.316897:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b033b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+63:2407:x1871938519997440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 368us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.316904:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100712 00000100:00100000:0.0:1785220582.316921:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100714 00000100:00100000:0.0:1785220582.316928:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+94:2407:x1871938519997952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.316935:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.316941:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007ff7ad65/0xf62499ab92793347 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f82c expref: 94 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316950:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007ff7ad65/0xf62499ab92793347 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f82c expref: 94 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316963:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7305:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.316966:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7305:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.316970:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316976:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.316990:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.317004:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+94:2407:x1871938519997952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (679us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.317010:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100714 00000100:00100000:0.0:1785220582.317024:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100715 00000100:00100000:0.0:1785220582.317031:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce79050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+62:2407:x1871938519998208:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.317038:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.317043:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ab9abaa4/0xf62499ab9279357e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa63 expref: 62 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317052:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ab9abaa4/0xf62499ab9279357e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa63 expref: 62 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317068:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab9abaa4/0xf62499ab9279357e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa63 expref: 62 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317077:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.317080:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x732e:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.317083:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+93:2407:x1871938519997184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 4469us (4780us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.317097:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317105:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.317112:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100705 00010000:00010000:0.0:1785220582.317122:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.317137:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce79050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+61:2407:x1871938519998208:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (793us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.317143:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100715 00010000:00010000:0.0:1785220582.317163:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317170:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317179:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317187:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317201:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220582.317248:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317257:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317265:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317271:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317279:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1785220582.317280:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+168:3653:x1871938510777600:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19719us (20157us total) trans 4295070792 rc 0/0 00000100:00100000:0.0:1785220582.317286:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284774 00010000:00010000:0.0:1785220582.317307:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1785220582.317312:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317320:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98cc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.317323:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317331:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317341:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317348:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd539a8f/0xf62499ab927a59e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.317352:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97a9b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+1362:3654:x1871938510777728:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20011us (20362us total) trans 4295070213 rc 0/0 00000100:00100000:1.0:1785220582.317357:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284771 00010000:00010000:0.0:1785220582.317363:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220582.317366:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317377:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.317380:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317387:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317396:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317397:0:14166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ff7ad65/0xf62499ab92793347 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f82c expref: 92 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317404:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000023df6f47/0xf62499ab927a59c5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7303:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.317408:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0001_UUID:14166:1871938510780160:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.317415:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.317419:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317423:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317425:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98dc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.317429:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317431:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317436:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317444:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317446:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317453:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317454:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a47a86f/0xf62499ab927a59b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.317460:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317465:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.317468:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.317469:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317473:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.317476:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317483:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317492:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317499:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a4d490c7/0xf62499ab927a599b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317511:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.317515:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317521:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.317524:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.317530:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97984700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+1362:3653:x1871938510779648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16113us (16283us total) trans 4295070214 rc 0/0 00010000:00010000:0.0:1785220582.317532:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.317537:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284780 00010000:00010000:0.0:1785220582.317542:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317549:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a59da lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7305:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.317560:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.317562:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.317563:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.317569:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.317604:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a98e00/2, rc 0, transno 4295070213 00010000:00010000:2.0:1785220582.317790:0:13820:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007e380aca/0xf62499ab92793339 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f81e expref: 91 pid: 14192 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.317802:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510780672:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.317817:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.317825:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.317833:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.317840:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.317847:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.317876:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.317953:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+1362:3654:x1871938510779136:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15676us (19595us total) trans 4295070215 rc 0/0 00000100:00100000:2.0:1785220582.317960:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284783 00010000:00010000:2.0:1785220582.317981:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.317986:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.317994:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.317997:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.318003:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.318013:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.318019:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a59ef lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7304:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.318043:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.318044:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.318047:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.318057:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.318216:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.318324:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.318373:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.318579:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.318590:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998464 00000100:00100000:1.0:1785220582.318595:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998464 00010000:00010000:1.0:1785220582.318618:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000526b0daf/0xf62499ab92793363 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f848 expref: 91 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.318641:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998720 00000100:00100000:1.0:1785220582.318644:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998720 00010000:00010000:1.0:1785220582.318659:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007c46a318/0xf62499ab92793371 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f856 expref: 92 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.318674:0:14211:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519998976 00000100:00100000:1.0:1785220582.318676:0:14211:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519998976 00010000:00010000:1.0:1785220582.318688:0:14211:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fd1b5920/0xf62499ab9279337f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f864 expref: 93 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.318707:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100716 00000100:00100000:1.0:1785220582.318715:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+93:2407:x1871938519998464:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.318725:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.318730:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000526b0daf/0xf62499ab92793363 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f848 expref: 93 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318740:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000526b0daf/0xf62499ab92793363 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f848 expref: 93 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318765:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000526b0daf/0xf62499ab92793363 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f848 expref: 93 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318778:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7307:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.318781:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7307:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.318786:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318794:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318815:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.318846:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+92:2407:x1871938519998464:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 132us (631us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.318856:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100716 00000100:00100000:1.0:1785220582.318870:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100717 00000100:00100000:1.0:1785220582.318876:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+91:2407:x1871938519998720:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.318883:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.318888:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007c46a318/0xf62499ab92793371 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f856 expref: 91 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318898:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007c46a318/0xf62499ab92793371 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f856 expref: 91 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318910:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007c46a318/0xf62499ab92793371 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f856 expref: 91 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318920:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7308:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.318920:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a98e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777728:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.318923:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7308:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.318927:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318934:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.318938:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100718 00000100:00100000:0.0:1785220582.318946:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+90:2407:x1871938519998976:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.318949:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.318954:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.318959:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fd1b5920/0xf62499ab9279337f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f864 expref: 90 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.318966:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+90:2407:x1871938519998720:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (644us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.318969:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fd1b5920/0xf62499ab9279337f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f864 expref: 90 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.318972:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100717 00010000:00010000:0.0:1785220582.318987:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd1b5920/0xf62499ab9279337f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f864 expref: 89 pid: 14164 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.318994:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.318996:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7309:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.318999:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7309:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.319004:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319005:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319013:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.319014:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.319022:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319031:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.319034:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.319049:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+88:2407:x1871938519998976:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 105us (676us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.319055:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100718 00010000:00010000:0.0:1785220582.319079:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319109:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.319152:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97985880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+1362:3654:x1871938510780288:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17259us (17481us total) trans 4295070216 rc 0/0 00000100:00100000:1.0:1785220582.319160:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284782 00010000:00010000:1.0:1785220582.319178:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.319184:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.319191:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98da00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.319195:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.319202:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.319217:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.319223:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a59e8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7307:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.319237:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.319239:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.319240:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.319259:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510781824:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.319280:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319289:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.319290:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.319334:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.319413:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97984e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1362:3653:x1871938510780800:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16554us (16610us total) trans 4295070217 rc 0/0 00000100:00100000:0.0:1785220582.319420:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284784 00000100:00100000:0.0:1785220582.319434:0:14247:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510781824 00000100:00100000:0.0:1785220582.319438:0:14247:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510781824 00000100:00100000:0.0:1785220582.319467:0:14247:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284788 00000100:00100000:0.0:1785220582.319473:0:14247:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1362:3652:x1871938510781824:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.319491:0:14247:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29450 count 1 00010000:00010000:0.0:1785220582.319510:0:14247:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319522:0:14247:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000095e82416/0xf62499ab9279338d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f872 expref: 87 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319529:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x730a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.319532:0:14247:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.319536:0:14247:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319556:0:14247:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000095e82416/0xf62499ab9279338d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f872 expref: 87 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319567:0:14247:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000095e82416/0xf62499ab9279338d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f872 expref: 87 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.319584:0:14247:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510781952:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.319788:0:14247:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.319807:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319816:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319825:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.319832:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.319880:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+60:2407:x1871938519997696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 3254us (3569us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.319888:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100713 00010000:00010000:3.0:1785220582.319941:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.319947:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.319951:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.319954:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.319959:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.319962:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.319966:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.319967:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.319973:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.319980:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.319985:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.319991:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b973d699/0xf62499ab927a59f6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320021:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.320025:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320031:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.320034:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320044:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320054:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320062:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.320065:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.320071:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97a9b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+168:3654:x1871938510777856:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22598us (23012us total) trans 4295070793 rc 0/0 00010000:00010000:2.0:1785220582.320071:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.320074:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.320077:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284773 00010000:00010000:2.0:1785220582.320082:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320111:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320130:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320143:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320152:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320159:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320173:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a59a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320175:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.320190:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.320194:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320201:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.320204:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320211:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320222:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320229:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a59d3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7306:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320243:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.320244:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94aa0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+1363:3654:x1871938510779776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18893us (19039us total) trans 4295070218 rc 0/0 00010000:00010000:2.0:1785220582.320244:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.320246:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.320250:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284779 00010000:00010000:2.0:1785220582.320252:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.320270:0:14259:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320277:0:14259:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320284:0:14259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320290:0:14259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.320306:0:14259:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.320370:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+168:3654:x1871938510779392:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19211us (19324us total) trans 4295070794 rc 0/0 00000100:00100000:3.0:1785220582.320376:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284778 00000100:00100000:2.0:1785220582.320410:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.320469:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.320475:0:14198:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a59fd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7309:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.320559:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+1363:3652:x1871938510781056:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15369us (15521us total) trans 4295070219 rc 0/0 00000100:00100000:0.0:1785220582.320566:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284785 00000100:00100000:0.0:1785220582.320600:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510781568:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.320746:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.320759:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320767:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.320771:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320782:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320793:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 1/0,0 mode: --/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320800:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b41e0fba/0xf62499ab927a59cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.320815:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.320817:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.320818:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.320919:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.320925:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.320949:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfc4ed80/2, rc 0, transno 4295070219 00000100:00100000:1.0:1785220582.320966:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfc4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510781056:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.321029:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.321039:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510782080:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.321041:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.321064:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519999104 00000100:00100000:1.0:1785220582.321068:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.321069:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519999104 00010000:00010000:0.0:1785220582.321107:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000923c2c8/0xf62499ab9279359a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa7f expref: 60 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.321145:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100719 00000100:00100000:2.0:1785220582.321196:0:13869:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510782080 00000100:00100000:2.0:1785220582.321200:0:13869:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510782080 00000100:00100000:2.0:1785220582.321228:0:13869:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284789 00000100:00100000:2.0:1785220582.321236:0:13869:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1363:3652:x1871938510782080:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.321257:0:13869:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29451 count 1 00010000:00010000:2.0:1785220582.321273:0:13869:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321286:0:13869:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008a196e47/0xf62499ab9279339b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f880 expref: 87 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321294:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x730b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.321298:0:13869:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.321302:0:13869:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321325:0:13869:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008a196e47/0xf62499ab9279339b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f880 expref: 87 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321337:0:13869:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008a196e47/0xf62499ab9279339b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f880 expref: 87 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.321348:0:13869:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510782208:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.321389:0:13869:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bae40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.321671:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+60:2407:x1871938519999104:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.321684:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.321690:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000923c2c8/0xf62499ab9279359a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa7f expref: 60 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.321702:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000923c2c8/0xf62499ab9279359a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa7f expref: 60 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.321717:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000923c2c8/0xf62499ab9279359a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa7f expref: 60 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.321728:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7330:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.321732:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7330:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.321736:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.321745:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.321785:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.321817:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+59:2407:x1871938519999104:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 664us (1407us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.321851:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321860:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321870:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321877:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.321891:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.321994:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90e8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+168:3654:x1871938510781312:12345-0@lo:6:osp-syn-0-0.0 Request processed in 9611us (9671us total) trans 4295070795 rc 0/0 00000100:00100000:2.0:1785220582.322004:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284786 00010000:00010000:2.0:1785220582.322022:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.322027:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.322035:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.322038:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.322046:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.322058:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.322066:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000547c73bb/0xf62499ab927a5a04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.322081:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.322107:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.322110:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.323428:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97986300/2, rc 0, transno 4295070217 00000100:00100000:1.0:1785220582.323453:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97986300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510780800:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.323468:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba6300/2, rc 0, transno 4295070791 00000100:00100000:1.0:1785220582.323485:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510779520:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.323496:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa2300/2, rc 0, transno 4295070214 00000100:00100000:1.0:1785220582.323499:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510779648:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.323508:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba7100/2, rc 0, transno 4295070212 00000100:00100000:1.0:1785220582.323511:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510779008:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.323521:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba6a00/2, rc 0, transno 4295070790 00000100:00100000:1.0:1785220582.323524:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510778752:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.323533:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a99880/2, rc 0, transno 4295070789 00000100:00100000:2.0:1785220582.323534:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90141500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782464:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.323537:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a99880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510777472:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.323544:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a99f80/2, rc 0, transno 4295070792 00000100:00100000:1.0:1785220582.323547:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a99f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510777600:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.323556:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1785220582.323561:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90141180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510782592:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.323575:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.323597:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.323681:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220582.323688:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90140380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782336:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.323711:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.323733:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782976:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.323741:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b032300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510782720:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.323765:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.323775:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510783104:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.323795:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.323798:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.323800:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782848:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.323819:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.323908:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510782464 00000100:00100000:3.0:1785220582.323914:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510782464 00000100:00100000:3.0:1785220582.323956:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510782592 00000100:00100000:3.0:1785220582.323960:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510782592 00000100:00100000:1.0:1785220582.323995:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.324207:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100719 00000100:00100000:2.0:1785220582.324594:0:13818:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510782336 00000100:00100000:2.0:1785220582.324598:0:13818:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510782336 00000100:00100000:1.0:1785220582.324604:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510782976 00000100:00100000:1.0:1785220582.324608:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510782976 00000100:00100000:1.0:1785220582.324630:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510783104 00000100:00100000:1.0:1785220582.324633:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510783104 00000100:00100000:1.0:1785220582.324648:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284790 00000100:00100000:1.0:1785220582.324656:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90140700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1366:3655:x1871938510782464:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.324681:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29453 count 1 00000100:00100000:2.0:1785220582.324691:0:13818:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510782848 00000100:00100000:2.0:1785220582.324694:0:13818:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510782848 00010000:00010000:1.0:1785220582.324705:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.324708:0:13818:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510782720 00000100:00100000:2.0:1785220582.324710:0:13818:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510782720 00000100:00100000:2.0:1785220582.324723:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284791 00000100:00100000:2.0:1785220582.324728:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1366:3653:x1871938510782592:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.324728:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000504140d6/0xf62499ab927933b7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f89c expref: 87 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.324737:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x730d:0x0].0x0 since=17s v=(0 0) 00002000:00080000:2.0:1785220582.324739:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29454 count 1 00010000:00010000:1.0:1785220582.324742:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.324746:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.324762:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.324773:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002e8503f5/0xf62499ab927933c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8aa expref: 87 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.324779:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x730e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.324782:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.324785:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.324788:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000504140d6/0xf62499ab927933b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f89c expref: 87 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.324800:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000504140d6/0xf62499ab927933b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f89c expref: 87 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.324801:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002e8503f5/0xf62499ab927933c5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8aa expref: 87 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.324813:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510783232:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.324815:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002e8503f5/0xf62499ab927933c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8aa expref: 87 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.324823:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510783360:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.324879:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.324941:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284792 00000100:00100000:3.0:1785220582.324950:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c928a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+171:3655:x1871938510782976:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.324962:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29492 count 1 00010000:00010000:3.0:1785220582.324975:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.324986:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002f6f96de/0xf62499ab927935d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab7 expref: 58 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.324993:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7334:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.324995:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7334:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.324999:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.325017:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002f6f96de/0xf62499ab927935d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab7 expref: 58 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.325026:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002f6f96de/0xf62499ab927935d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab7 expref: 58 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.325037:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510783488:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.325060:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.325065:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.325107:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510781696:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.325141:0:13869:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_011:lustre-OST0001_UUID:13869:1871938510782208:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.325157:0:13869:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.326024:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.326053:0:14150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284793 00000100:00100000:1.0:1785220582.326063:0:14150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfc4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+171:3652:x1871938510783104:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.326083:0:14150:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29493 count 1 00000100:00100000:0.0:1785220582.326580:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284794 00000100:00100000:0.0:1785220582.326593:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1366:3655:x1871938510782336:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.326604:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.326618:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29452 count 1 00010000:00010000:0.0:1785220582.326634:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.326648:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e1d73704/0xf62499ab927933a9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f88e expref: 87 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.326654:0:14150:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.326656:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x730c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.326660:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.326664:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.326666:0:14150:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009c6b137d/0xf62499ab927935e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fac5 expref: 58 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.326678:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7335:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.326680:0:14150:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7335:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.326683:0:14150:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.326690:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e1d73704/0xf62499ab927933a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f88e expref: 87 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.326700:0:14150:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009c6b137d/0xf62499ab927935e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fac5 expref: 58 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.326701:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e1d73704/0xf62499ab927933a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f88e expref: 87 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.326710:0:14150:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009c6b137d/0xf62499ab927935e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fac5 expref: 58 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.326721:0:14150:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510783744:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.326721:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510783616:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.326765:0:14150:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896a80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.326784:0:14170:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284795 00000100:00100000:1.0:1785220582.326793:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c928700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+171:3655:x1871938510782848:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.326806:0:14170:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29491 count 1 00010000:00010000:1.0:1785220582.326818:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.327370:0:14170:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000602a2c62/0xf62499ab927935c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa9 expref: 58 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.327378:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7333:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.327381:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7333:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.327386:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.327415:0:14170:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000602a2c62/0xf62499ab927935c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa9 expref: 58 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.327425:0:14170:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000602a2c62/0xf62499ab927935c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa9 expref: 58 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.327438:0:14170:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510783872:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.327445:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.327461:0:14170:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.327467:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284796 00000100:00100000:0.0:1785220582.327477:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+171:3653:x1871938510782720:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.327491:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29490 count 1 00010000:00010000:0.0:1785220582.327504:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327516:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4e61da1/0xf62499ab927935b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa9b expref: 58 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327522:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7332:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.327526:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7332:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.327529:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327548:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d4e61da1/0xf62499ab927935b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa9b expref: 58 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327556:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d4e61da1/0xf62499ab927935b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa9b expref: 58 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.327566:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0000_UUID:14260:1871938510784000:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.327591:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f480 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.327724:0:14247:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_073:lustre-OST0001_UUID:14247:1871938510781952:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.327746:0:14247:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.327778:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519999232 00000100:00100000:0.0:1785220582.327783:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519999232 00010000:00010000:0.0:1785220582.327811:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000045cbe019/0xf62499ab927935a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa8d expref: 59 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.327847:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100720 00000100:00100000:0.0:1785220582.327855:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9147bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+59:2406:x1871938519999232:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.327864:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.327870:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000045cbe019/0xf62499ab927935a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa8d expref: 59 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327880:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000045cbe019/0xf62499ab927935a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa8d expref: 59 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327898:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045cbe019/0xf62499ab927935a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa8d expref: 59 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327911:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7331:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.327914:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7331:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.327919:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327928:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.327954:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.327982:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.327982:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+58:2406:x1871938519999232:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 128us (2922us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.327990:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100720 00010000:00010000:3.0:1785220582.327991:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.328000:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.328008:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.328030:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1785220582.328040:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8f480/2, rc 0, transno 4295070795 00000100:00100000:0.0:1785220582.328060:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510781312:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.328074:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97985180/2, rc 0, transno 4295070216 00000100:00100000:0.0:1785220582.328173:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97985180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510780288:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.328225:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000004:00080000:0.0:1785220582.328230:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94aa3b80/2, rc 0, transno 4295070218 00000100:00100000:1.0:1785220582.328236:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784128:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.328236:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94aa3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510779776:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.328267:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba5180/2, rc 0, transno 4295070794 00000100:00100000:0.0:1785220582.328274:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510779392:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.328276:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.328296:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784256:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.328299:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba6d80/2, rc 0, transno 4295070215 00000100:00100000:0.0:1785220582.328305:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510779136:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.328318:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.328330:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97a9aa00/2, rc 0, transno 4295070793 00000100:00100000:0.0:1785220582.328336:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97a9aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510777856:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.328338:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784640:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.328357:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.328360:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510784384:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.328373:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.328378:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784768:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.328382:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.328394:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.328399:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.328404:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510784512:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.328425:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.328558:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90e8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+172:3654:x1871938510781440:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12674us (13660us total) trans 4295070796 rc 0/0 00000100:00100000:3.0:1785220582.328565:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284787 00000100:00100000:3.0:1785220582.328582:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510784128 00000100:00100000:3.0:1785220582.328585:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510784128 00000100:00100000:3.0:1785220582.328606:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510784256 00000100:00100000:3.0:1785220582.328609:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510784256 00000100:00100000:3.0:1785220582.328622:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510784640 00000100:00100000:3.0:1785220582.328624:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510784640 00000100:00100000:3.0:1785220582.328636:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510784384 00000100:00100000:3.0:1785220582.328638:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510784384 00000100:00100000:3.0:1785220582.328651:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510784768 00000100:00100000:3.0:1785220582.328654:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510784768 00000100:00100000:3.0:1785220582.328666:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510784512 00000100:00100000:3.0:1785220582.328669:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510784512 00000100:00100000:3.0:1785220582.328689:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284797 00000100:00100000:3.0:1785220582.328695:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d99233c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1369:3652:x1871938510784128:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.328710:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29455 count 1 00010000:00010000:3.0:1785220582.328726:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.328740:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b16d8bb7/0xf62499ab927933d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b8 expref: 87 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.328759:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x730f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.328763:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.328766:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.328786:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b16d8bb7/0xf62499ab927933d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b8 expref: 87 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.328795:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b16d8bb7/0xf62499ab927933d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b8 expref: 87 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.328810:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510784896:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.328921:0:14222:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284798 00000100:00100000:1.0:1785220582.328930:0:14222:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+174:3652:x1871938510784256:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.328942:0:14222:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29494 count 1 00010000:00010000:1.0:1785220582.328953:0:14222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.328964:0:14222:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000650f0f06/0xf62499ab927935ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fad3 expref: 57 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.328970:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7336:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.328973:0:14222:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7336:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.328976:0:14222:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.328996:0:14222:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000650f0f06/0xf62499ab927935ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fad3 expref: 57 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329006:0:14222:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000650f0f06/0xf62499ab927935ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fad3 expref: 57 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.329015:0:14222:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510785024:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.329052:0:14222:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896900 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.329070:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284799 00000100:00100000:1.0:1785220582.329078:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1369:3652:x1871938510784640:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.329124:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29457 count 1 00010000:00010000:1.0:1785220582.329137:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329147:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000edce6884/0xf62499ab927933ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8d4 expref: 87 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329157:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7311:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.329160:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7311:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.329164:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329184:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000edce6884/0xf62499ab927933ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8d4 expref: 87 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329193:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000edce6884/0xf62499ab927933ef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8d4 expref: 87 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.329203:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510785152:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.329230:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.329246:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284800 00000100:00100000:1.0:1785220582.329253:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1369:3654:x1871938510784384:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.329263:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29456 count 1 00010000:00010000:1.0:1785220582.329275:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329285:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001a4975db/0xf62499ab927933e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8c6 expref: 87 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329291:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7310:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.329294:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7310:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.329298:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329317:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001a4975db/0xf62499ab927933e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8c6 expref: 87 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329326:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001a4975db/0xf62499ab927933e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8c6 expref: 87 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.329336:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510785280:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.329357:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.329374:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284801 00000100:00100000:1.0:1785220582.329381:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d998b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+174:3652:x1871938510784768:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.329393:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29496 count 1 00010000:00010000:1.0:1785220582.329403:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329416:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001062cda2/0xf62499ab9279360a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faef expref: 57 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329422:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7338:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.329425:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7338:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.329429:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329450:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001062cda2/0xf62499ab9279360a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faef expref: 57 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.329460:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001062cda2/0xf62499ab9279360a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faef expref: 57 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.329470:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae92680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510785408:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.329493:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.329577:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.329596:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284802 00000100:00100000:3.0:1785220582.329605:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+174:3654:x1871938510784512:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.329617:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29495 count 1 00010000:00010000:3.0:1785220582.329630:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.329640:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009e755ee1/0xf62499ab927935fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae1 expref: 57 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.329646:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7337:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.329649:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7337:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.329653:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.329671:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009e755ee1/0xf62499ab927935fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae1 expref: 57 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.329680:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009e755ee1/0xf62499ab927935fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae1 expref: 57 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.329689:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510785536:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.329713:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c840 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.331405:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8c380/2, rc 0, transno 4295070796 00000100:00100000:0.0:1785220582.331423:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510781440:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.331487:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97987480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510785664:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.331528:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.331627:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.331635:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.331646:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.331650:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.331661:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.331682:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.331690:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5a0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.331707:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.331708:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.331710:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.331716:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510785664 00000100:00100000:2.0:1785220582.331723:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510785664 00000100:00100000:2.0:1785220582.331778:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284803 00000100:00100000:2.0:1785220582.331787:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97985c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+175:3654:x1871938510785664:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.331806:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29497 count 1 00010000:00010000:2.0:1785220582.331824:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.331840:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000448bc592/0xf62499ab92793618 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fafd expref: 57 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.331848:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7339:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.331851:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7339:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.331856:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.331882:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000448bc592/0xf62499ab92793618 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fafd expref: 57 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.331895:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000448bc592/0xf62499ab92793618 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fafd expref: 57 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.331906:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d91479180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510785792:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.331950:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.331978:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.332045:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519999360 00000100:00100000:0.0:1785220582.332050:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519999360 00010000:00010000:0.0:1785220582.332073:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008a196e47/0xf62499ab9279339b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f880 expref: 88 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.332124:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100721 00000100:00100000:0.0:1785220582.332132:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9147a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+88:2407:x1871938519999360:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.332142:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.332147:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008a196e47/0xf62499ab9279339b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f880 expref: 88 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332158:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008a196e47/0xf62499ab9279339b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f880 expref: 88 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332175:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a196e47/0xf62499ab9279339b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f880 expref: 88 pid: 14160 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332187:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.332191:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.332198:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332205:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332235:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.332256:0:13869:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.332258:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9147a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+87:2407:x1871938519999360:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 127us (280us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.332266:0:13869:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.332266:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100721 00010000:00010000:1.0:1785220582.332274:0:13869:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.332282:0:13869:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.332305:0:13869:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.332460:0:13869:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+1370:3652:x1871938510782080:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11226us (11391us total) trans 4295070220 rc 0/0 00000100:00100000:1.0:1785220582.332469:0:13869:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284789 00000004:00080000:1.0:1785220582.332510:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89eff800/2, rc 0, transno 4295070220 00000100:00100000:1.0:1785220582.332519:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510782080:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.332693:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.332755:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332764:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.332767:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332775:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332787:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332795:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f228b1ca/0xf62499ab927a5a19 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.332816:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.332818:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.332820:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.332878:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.332889:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90ba7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510785920:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.332917:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.333082:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.333112:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510785920 00000100:00100000:1.0:1785220582.333125:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510785920 00000100:00100000:1.0:1785220582.333153:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284804 00000100:00100000:2.0:1785220582.333158:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510783360:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.333160:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d90ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1370:3654:x1871938510785920:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.333175:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.333175:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29458 count 1 00010000:00010000:1.0:1785220582.333190:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.333202:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.333203:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007bad18e6/0xf62499ab927933fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e2 expref: 86 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.333211:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7312:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.333215:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7312:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.333219:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.333230:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.333240:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007bad18e6/0xf62499ab927933fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e2 expref: 86 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.333252:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007bad18e6/0xf62499ab927933fd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e2 expref: 86 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.333263:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510786048:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.333320:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519999488 00000100:00100000:3.0:1785220582.333325:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519999488 00000100:00100000:0.0:1785220582.333329:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510783616:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.333343:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.333344:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000095e82416/0xf62499ab9279338d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f872 expref: 87 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.333370:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519999616 00000100:00100000:0.0:1785220582.333371:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519999744 00000100:00100000:3.0:1785220582.333373:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519999616 00000100:00100000:0.0:1785220582.333373:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519999744 00010000:00010000:3.0:1785220582.333385:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002e8503f5/0xf62499ab927933c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8aa expref: 88 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.333387:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002f6f96de/0xf62499ab927935d2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab7 expref: 58 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.333441:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100722 00000100:00100000:3.0:1785220582.333449:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9aec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+88:2406:x1871938519999488:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.333460:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.333464:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000095e82416/0xf62499ab9279338d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f872 expref: 88 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.333474:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000095e82416/0xf62499ab9279338d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f872 expref: 88 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.333487:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095e82416/0xf62499ab9279338d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f872 expref: 88 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.333497:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.333500:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.333505:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.333512:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.333529:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.334383:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100723 00000100:00100000:0.0:1785220582.334395:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d91478700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+87:2406:x1871938519999616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.334407:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.334414:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002e8503f5/0xf62499ab927933c5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8aa expref: 87 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.334426:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002e8503f5/0xf62499ab927933c5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8aa expref: 87 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.334447:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002e8503f5/0xf62499ab927933c5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8aa expref: 87 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.334458:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.334461:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.334466:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.334474:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.334503:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.334563:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.334593:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334603:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334624:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334633:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334649:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.334808:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+1371:3653:x1871938510782592:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10078us (11209us total) trans 4295070221 rc 0/0 00000100:00100000:2.0:1785220582.334816:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284791 00010000:00010000:2.0:1785220582.334840:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.334846:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334854:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.334857:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334866:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334878:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334885:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5a27 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.334900:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.334902:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.334904:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.334915:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.334937:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000000 00000100:00100000:1.0:1785220582.334942:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000000 00010000:00010000:1.0:1785220582.334963:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009c6b137d/0xf62499ab927935e0 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fac5 expref: 59 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.334993:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100724 00000100:00100000:1.0:1785220582.335000:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+59:2406:x1871938519999744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.335009:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220582.335014:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.335014:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002f6f96de/0xf62499ab927935d2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab7 expref: 59 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.335024:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002f6f96de/0xf62499ab927935d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab7 expref: 59 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.335033:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.335038:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7334:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.335041:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7334:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.335046:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.335054:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.335067:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.335074:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.335115:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.335540:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9aec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+86:2406:x1871938519999488:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2090us (2456us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.335549:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100722 00000100:00100000:3.0:1785220582.335595:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938519999872 00000100:00100000:3.0:1785220582.335600:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938519999872 00010000:00010000:3.0:1785220582.335616:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000504140d6/0xf62499ab927933b7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f89c expref: 86 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.335637:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000128 00000100:00100000:3.0:1785220582.335641:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000128 00010000:00010000:3.0:1785220582.335655:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e1d73704/0xf62499ab927933a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f88e expref: 87 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.335672:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000256 00000100:00100000:3.0:1785220582.335675:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000256 00010000:00010000:3.0:1785220582.335689:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000602a2c62/0xf62499ab927935c4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa9 expref: 60 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.335705:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000384 00000100:00100000:3.0:1785220582.335707:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000384 00010000:00010000:3.0:1785220582.335719:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d4e61da1/0xf62499ab927935b6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa9b expref: 61 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.335732:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100725 00000100:00100000:3.0:1785220582.335739:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+61:2406:x1871938520000000:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.335758:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.335764:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009c6b137d/0xf62499ab927935e0 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fac5 expref: 61 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.335779:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009c6b137d/0xf62499ab927935e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fac5 expref: 61 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.335795:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7335:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.335798:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7335:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.335803:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.335813:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.335853:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100726 00000100:00100000:2.0:1785220582.335862:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+87:2406:x1871938519999872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.335872:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.335877:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000504140d6/0xf62499ab927933b7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f89c expref: 87 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.335886:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000504140d6/0xf62499ab927933b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f89c expref: 87 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.335899:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.335902:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.335906:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.335913:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.335928:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.335948:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+87:2406:x1871938519999872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 88us (934us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.335954:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100726 00000100:00100000:2.0:1785220582.335970:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100727 00000100:00100000:2.0:1785220582.335976:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+86:2406:x1871938520000128:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.335983:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.335988:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e1d73704/0xf62499ab927933a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f88e expref: 86 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.335997:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e1d73704/0xf62499ab927933a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f88e expref: 86 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336010:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e1d73704/0xf62499ab927933a9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f88e expref: 86 pid: 14188 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336021:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.336024:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.336028:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336036:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336054:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.336072:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+85:2406:x1871938520000128:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1037us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.336079:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100727 00000100:00100000:2.0:1785220582.336113:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100728 00000100:00100000:2.0:1785220582.336120:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+61:2406:x1871938520000256:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.336128:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.336133:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000602a2c62/0xf62499ab927935c4 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa9 expref: 61 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336141:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000602a2c62/0xf62499ab927935c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa9 expref: 61 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336152:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7333:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.336155:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7333:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.336159:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336167:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336179:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.336196:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+61:2406:x1871938520000256:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (1121us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.336203:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100728 00000100:00100000:1.0:1785220582.336213:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+60:2406:x1871938519999744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1212us (2981us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.336216:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100729 00000100:00100000:1.0:1785220582.336221:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100724 00000100:00100000:2.0:1785220582.336222:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90700450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+60:2407:x1871938520000384:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.336229:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.336234:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d4e61da1/0xf62499ab927935b6 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa9b expref: 59 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336248:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d4e61da1/0xf62499ab927935b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa9b expref: 59 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336258:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7332:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.336261:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7332:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.336264:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.336266:0:13908:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000504140d6/0xf62499ab927933b7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f89c expref: 84 pid: 14158 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336272:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.336283:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510783232:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.336284:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.336298:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90700450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+59:2407:x1871938520000384:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 78us (1184us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.336301:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.336304:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100729 00010000:00010000:1.0:1785220582.336310:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.336317:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336322:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.336327:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336329:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.336336:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336338:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336345:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.336351:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.336359:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220582.336539:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.336560:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+58:2406:x1871938520000000:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 821us (1996us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.336566:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100725 00010000:00010000:3.0:1785220582.336602:0:13866:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f6f96de/0xf62499ab927935d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab7 expref: 57 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.336614:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:13866:1871938510783488:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.336626:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.336634:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.336645:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.336654:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.336662:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.336678:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:3.0:1785220582.336798:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c928a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+176:3655:x1871938510782976:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11850us (13031us total) trans 4295070797 rc 0/0 00000100:00100000:3.0:1785220582.336806:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284792 00000100:00100000:2.0:1785220582.336955:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90143480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+1372:3655:x1871938510782336:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10365us (13241us total) trans 4295070222 rc 0/0 00000100:00100000:2.0:1785220582.336961:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284794 00010000:00010000:2.0:1785220582.336982:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.336989:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.336996:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.337001:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337009:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337020:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337027:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008d120fb0/0xf62499ab927a5a20 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337045:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.337049:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337061:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98dc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.337064:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337072:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337082:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337114:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5a3c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337128:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.337131:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337137:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.337140:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337147:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337157:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337165:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f5c04ab/0xf62499ab927a5a2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.337179:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.337181:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.337183:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.337422:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90140700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+1371:3655:x1871938510782464:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12766us (13845us total) trans 4295070223 rc 0/0 00000100:00100000:1.0:1785220582.337428:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284790 00000100:00100000:1.0:1785220582.337461:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510785152:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.337477:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.337511:0:14150:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c6b137d/0xf62499ab927935e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fac5 expref: 56 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.337521:0:14150:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:14150:1871938510783744:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.337531:0:14150:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.337538:0:14150:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.337544:0:14150:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.337552:0:14150:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.337559:0:14150:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.338032:0:14150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.338369:0:14150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+176:3652:x1871938510783104:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12307us (14569us total) trans 4295070798 rc 0/0 00000100:00100000:1.0:1785220582.338377:0:14150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284793 00000100:00100000:1.0:1785220582.338413:0:14222:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_068:lustre-OST0000_UUID:14222:1871938510785024:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.338428:0:14222:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.338454:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000602a2c62/0xf62499ab927935c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa9 expref: 55 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.338465:0:14170:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_031:lustre-OST0000_UUID:14170:1871938510783872:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.338472:0:14260:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4e61da1/0xf62499ab927935b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa9b expref: 54 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.338476:0:14170:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.338484:0:14170:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.338484:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0000_UUID:14260:1871938510784000:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.338491:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.338497:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.338500:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.338505:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.338508:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.338512:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.338522:0:14170:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220582.338524:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.338531:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.338538:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220582.338591:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c928700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+177:3655:x1871938510782848:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11799us (14770us total) trans 4295070799 rc 0/0 00000100:00100000:0.0:1785220582.338593:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+177:3653:x1871938510782720:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11120us (14599us total) trans 4295070800 rc 0/0 00000100:00100000:0.0:1785220582.338600:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284796 00010000:00010000:2.0:1785220582.338722:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.338729:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338738:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.338741:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338758:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338773:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338780:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cbb6036/0xf62499ab927a5a35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338798:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.338801:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338808:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.338811:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338818:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338827:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338834:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d95ab63b/0xf62499ab927a5a43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338846:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.338849:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338856:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.338859:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338867:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338876:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338884:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5a4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.338898:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.338899:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.338901:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.338907:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.339327:0:14170:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284795 00000100:00100000:1.0:1785220582.339369:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510785280:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.339385:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.339413:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae92680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510785408:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.339429:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.339480:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b033b80/2, rc 0, transno 4295070798 00000100:00100000:1.0:1785220582.339495:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510783104:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.339519:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b031500/2, rc 0, transno 4295070799 00000100:00100000:1.0:1785220582.339522:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782848:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.339522:0:14247:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.339531:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b031180/2, rc 0, transno 4295070797 00010000:00010000:0.0:1785220582.339531:0:14247:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.339534:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782976:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.339540:0:14247:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.339545:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90140380/2, rc 0, transno 4295070222 00010000:00010000:0.0:1785220582.339548:0:14247:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.339554:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90140380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782336:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.339563:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90141500/2, rc 0, transno 4295070223 00010000:00010000:0.0:1785220582.339566:0:14247:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.339567:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510782464:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.339884:0:14247:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfc4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_073:lustre-MDT0000-mdtlov_UUID+1371:3652:x1871938510781824:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20411us (20592us total) trans 4295070224 rc 0/0 00000100:00100000:0.0:1785220582.339894:0:14247:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284788 00010000:00010000:0.0:1785220582.339915:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.339919:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.339927:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.339930:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.339937:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.339950:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.339957:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000c386a97/0xf62499ab927a5a12 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.339973:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1785220582.339975:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efc000/2, rc 0, transno 4295070224 00010000:00010000:0.0:1785220582.339975:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.339977:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.339980:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510781824:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.340056:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.340075:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.340077:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510784896:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.340171:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d91478700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+83:2406:x1871938519999616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 5778us (6966us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.340180:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100723 00000004:00080000:1.0:1785220582.340250:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b032300/2, rc 0, transno 4295070800 00000100:00100000:1.0:1785220582.340255:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b032300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510782720:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1785220582.340264:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90141180/2, rc 0, transno 4295070221 00000100:00100000:1.0:1785220582.340268:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90141180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510782592:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.340280:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510786304:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.340311:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.340337:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510786304 00000100:00100000:0.0:1785220582.340338:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510786432:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.340342:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510786304 00000100:00100000:0.0:1785220582.340361:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.340472:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.340502:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510785536:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.340514:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.340829:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.340850:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.340873:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.340889:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001280 00000100:00100000:3.0:1785220582.340893:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001280 00000100:00100000:0.0:1785220582.340895:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510786944:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.340914:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000448bc592/0xf62499ab92793618 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fafd expref: 54 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.340918:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.340936:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000512 00000100:00100000:3.0:1785220582.340939:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000512 00010000:00010000:3.0:1785220582.340953:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000edce6884/0xf62499ab927933ef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8d4 expref: 83 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.340958:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d91479180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0000_UUID:14189:1871938510785792:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.340970:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000768 00000100:00100000:3.0:1785220582.340973:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000768 00010000:00010000:2.0:1785220582.340974:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.340983:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001a4975db/0xf62499ab927933e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8c6 expref: 84 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.340995:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100730 00000100:00100000:3.0:1785220582.340999:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100731 00000100:00100000:2.0:1785220582.341003:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+54:2405:x1871938520001280:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.341005:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+84:2406:x1871938520000512:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.341015:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.341015:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.341020:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000edce6884/0xf62499ab927933ef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8d4 expref: 84 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.341021:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000448bc592/0xf62499ab92793618 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fafd expref: 54 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.341032:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000edce6884/0xf62499ab927933ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8d4 expref: 84 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.341033:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000448bc592/0xf62499ab92793618 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fafd expref: 54 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.341051:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000448bc592/0xf62499ab92793618 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fafd expref: 54 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.341052:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000edce6884/0xf62499ab927933ef lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8d4 expref: 84 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.341072:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7311:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.341072:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7339:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.341076:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7311:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.341076:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7339:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.341082:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.341083:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.341204:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.341216:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.341226:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.341230:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.341262:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+53:2405:x1871938520001280:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 256us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.341269:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100730 00000100:00100000:1.0:1785220582.341283:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510786432 00000100:00100000:2.0:1785220582.341285:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100732 00000100:00100000:1.0:1785220582.341287:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510786432 00000100:00100000:2.0:1785220582.341292:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+83:2406:x1871938520000768:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.341300:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.341304:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001a4975db/0xf62499ab927933e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8c6 expref: 83 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.341309:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284805 00010000:00010000:2.0:1785220582.341315:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001a4975db/0xf62499ab927933e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8c6 expref: 83 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.341315:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8ae92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1372:3653:x1871938510786304:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.341328:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001a4975db/0xf62499ab927933e1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8c6 expref: 83 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1785220582.341337:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29460 count 1 00010000:00010000:2.0:1785220582.341342:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7310:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.341345:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7310:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.341348:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.341354:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.341355:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.341367:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0c30762/0xf62499ab92793419 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8fe expref: 82 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.341373:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.341374:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7314:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.341377:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7314:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.341380:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.341437:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1785220582.341444:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510786560:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.341467:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.341483:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510787072:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.341503:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.341527:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d90e8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510786176:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.341547:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.341566:0:14223:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510786944 00000100:00100000:0.0:1785220582.341570:0:14223:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510786944 00000100:00100000:0.0:1785220582.341592:0:14223:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510786560 00000100:00100000:0.0:1785220582.341596:0:14223:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510786560 00000100:00100000:0.0:1785220582.341688:0:14223:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510787072 00000100:00100000:0.0:1785220582.341691:0:14223:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510787072 00000100:00100000:0.0:1785220582.341709:0:14223:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510786176 00000100:00100000:0.0:1785220582.341711:0:14223:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510786176 00000100:00100000:0.0:1785220582.341727:0:14223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284806 00000100:00100000:0.0:1785220582.341733:0:14223:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d8af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+1374:3654:x1871938510786432:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.341757:0:14223:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29461 count 1 00010000:00010000:0.0:1785220582.341770:0:14223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.341782:0:14223:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005a28fa1f/0xf62499ab92793427 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f90c expref: 82 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.341789:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7315:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.341792:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7315:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.341795:0:14223:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.341833:0:14223:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005a28fa1f/0xf62499ab92793427 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f90c expref: 82 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.341846:0:14223:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005a28fa1f/0xf62499ab92793427 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f90c expref: 82 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.341859:0:14223:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0001_UUID:14223:1871938510787328:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.341911:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e0c30762/0xf62499ab92793419 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8fe expref: 82 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.341922:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e0c30762/0xf62499ab92793419 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8fe expref: 82 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.341933:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510787200:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.342067:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+82:2406:x1871938520000768:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 773us (1191us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.342074:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100732 00010000:00010000:2.0:1785220582.342125:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342134:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342143:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342151:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342168:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.342264:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+81:2406:x1871938520000512:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1257us (1411us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.342271:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100731 00000100:00100000:3.0:1785220582.342293:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284807 00000100:00100000:3.0:1785220582.342300:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d8ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+178:3654:x1871938510786944:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.342313:0:8440:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29500 count 1 00000100:00100000:2.0:1785220582.342314:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97985c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+178:3654:x1871938510785664:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10529us (10785us total) trans 4295070801 rc 0/0 00000100:00100000:2.0:1785220582.342320:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284803 00010000:00010000:3.0:1785220582.342325:0:8440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.342335:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284808 00010000:00010000:3.0:1785220582.342336:0:8440:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000db74cbba/0xf62499ab92793642 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb27 expref: 52 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.342342:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d8ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1374:3654:x1871938510786560:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.342344:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x733c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.342347:0:8440:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.342351:0:8440:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.342352:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29462 count 1 00010000:00010000:2.0:1785220582.342361:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342371:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005ada6d94/0xf62499ab92793435 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f91a expref: 80 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342375:0:8440:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000db74cbba/0xf62499ab92793642 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb27 expref: 52 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342378:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7316:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.342381:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7316:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.342385:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342386:0:8440:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000db74cbba/0xf62499ab92793642 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb27 expref: 52 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342397:0:8440:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510787456:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.342401:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005ada6d94/0xf62499ab92793435 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f91a expref: 80 pid: 14185 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342409:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005ada6d94/0xf62499ab92793435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f91a expref: 80 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.342418:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510787584:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.342430:0:8440:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c840 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.342435:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2ba180 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.342448:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284809 00000100:00100000:2.0:1785220582.342455:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d8afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+177:3654:x1871938510787072:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.342461:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00002000:00080000:2.0:1785220582.342466:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29501 count 1 00010000:00010000:2.0:1785220582.342476:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342487:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc2f525c/0xf62499ab92793650 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb35 expref: 52 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342493:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x733d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.342496:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.342500:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342513:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.342516:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cc2f525c/0xf62499ab92793650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb35 expref: 52 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342525:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cc2f525c/0xf62499ab92793650 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb35 expref: 52 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.342535:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510787712:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.342539:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.342557:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a2bab40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.342587:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284810 00000100:00100000:3.0:1785220582.342589:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.342593:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfad0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1374:3652:x1871938510786176:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.342605:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29459 count 1 00000100:00100000:3.0:1785220582.342634:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.342652:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001024 00000100:00100000:3.0:1785220582.342656:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001024 00010000:00010000:3.0:1785220582.342675:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b16d8bb7/0xf62499ab927933d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b8 expref: 81 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342697:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000640 00010000:00010000:2.0:1785220582.342699:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342700:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000640 00010000:00010000:2.0:1785220582.342711:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb3ab728/0xf62499ab9279340b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f0 expref: 81 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342712:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000650f0f06/0xf62499ab927935ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fad3 expref: 53 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342719:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7313:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.342722:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7313:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.342725:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342727:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520000896 00000100:00100000:3.0:1785220582.342729:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520000896 00010000:00010000:3.0:1785220582.342740:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001062cda2/0xf62499ab9279360a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faef expref: 54 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342744:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eb3ab728/0xf62499ab9279340b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f0 expref: 81 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342766:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eb3ab728/0xf62499ab9279340b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f0 expref: 81 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342775:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001152 00000100:00100000:2.0:1785220582.342776:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510787840:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.342777:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001152 00010000:00010000:3.0:1785220582.342789:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009e755ee1/0xf62499ab927935fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae1 expref: 55 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.342798:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.342804:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001408 00000100:00100000:3.0:1785220582.342807:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001408 00010000:00010000:2.0:1785220582.342810:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.342816:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342818:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007bad18e6/0xf62499ab927933fd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e2 expref: 82 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342835:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100733 00000100:00100000:3.0:1785220582.342841:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+82:2406:x1871938520001024:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.342849:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.342854:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b16d8bb7/0xf62499ab927933d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b8 expref: 82 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342864:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b16d8bb7/0xf62499ab927933d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b8 expref: 82 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342882:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b16d8bb7/0xf62499ab927933d3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8b8 expref: 82 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342896:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.342899:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x730f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.342903:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342910:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342929:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.342935:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.342939:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342946:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+81:2406:x1871938520001024:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 106us (485us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.342953:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100733 00010000:00010000:2.0:1785220582.342954:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342966:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100734 00010000:00010000:2.0:1785220582.342967:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.342972:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+55:2406:x1871938520000640:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.342975:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b499a259/0xf62499ab927a5a7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342979:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.342983:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000650f0f06/0xf62499ab927935ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fad3 expref: 55 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.342992:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000650f0f06/0xf62499ab927935ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fad3 expref: 55 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.342992:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.342994:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.342996:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.343004:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000650f0f06/0xf62499ab927935ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fad3 expref: 55 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.343006:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100735 00000100:00100000:2.0:1785220582.343013:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+54:2406:x1871938520000896:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.343014:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7336:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.343017:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7336:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.343020:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.343021:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343025:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001062cda2/0xf62499ab9279360a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faef expref: 54 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343030:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343034:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001062cda2/0xf62499ab9279360a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faef expref: 54 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343045:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001062cda2/0xf62499ab9279360a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faef expref: 54 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343046:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.343060:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+53:2406:x1871938520000640:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 89us (547us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.343060:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7338:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.343063:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7338:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.343066:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100734 00010000:00010000:2.0:1785220582.343067:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343074:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.343078:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100736 00000100:00100000:0.0:1785220582.343177:0:14223:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.343204:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100737 00000100:00100000:0.0:1785220582.343212:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+80:2406:x1871938520001408:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.343220:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.343225:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007bad18e6/0xf62499ab927933fd lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e2 expref: 80 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343234:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007bad18e6/0xf62499ab927933fd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e2 expref: 80 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343247:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7312:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.343250:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7312:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.343254:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343262:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.343273:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac540c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+52:2406:x1871938520001152:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.343278:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.343282:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.343288:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009e755ee1/0xf62499ab927935fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae1 expref: 52 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.343295:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+80:2406:x1871938520001408:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (662us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.343297:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009e755ee1/0xf62499ab927935fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae1 expref: 52 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.343302:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100737 00010000:00010000:3.0:1785220582.343310:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e755ee1/0xf62499ab927935fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae1 expref: 52 pid: 14257 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343320:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7337:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.343323:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7337:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.343327:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343339:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.343352:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97987480/2, rc 0, transno 4295070801 00010000:00010000:3.0:1785220582.343354:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.343363:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510785664:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.343370:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac540c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+51:2406:x1871938520001152:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 288us (781us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.343376:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100736 00010000:00010000:3.0:1785220582.343394:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343402:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343411:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.343414:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510786688:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1785220582.343419:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343440:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.343446:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.343465:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.343469:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343476:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.343479:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343486:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343497:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343503:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b7f64e58/0xf62499ab927a5a51 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x730f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.343523:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.343525:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.343527:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.343541:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510787968:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.343543:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d99233c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+1375:3652:x1871938510784128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14848us (15267us total) trans 4295070225 rc 0/0 00000100:00100000:3.0:1785220582.343550:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284797 00000100:00100000:0.0:1785220582.343561:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.343583:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8c700/2, rc 0, transno 4295070225 00000100:00100000:0.0:1785220582.343590:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784128:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.343640:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510786688 00000100:00100000:3.0:1785220582.343644:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510786688 00000100:00100000:3.0:1785220582.343665:0:14156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510787968 00000100:00100000:3.0:1785220582.343668:0:14156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510787968 00000100:00100000:3.0:1785220582.343684:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284811 00000100:00100000:3.0:1785220582.343690:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfad1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+179:3652:x1871938510786688:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.343702:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29498 count 1 00010000:00010000:2.0:1785220582.343709:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.343715:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.343725:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+50:2406:x1871938520000896:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 713us (1185us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.343726:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007f130146/0xf62499ab92793626 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb0b expref: 50 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.343731:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100735 00010000:00010000:3.0:1785220582.343733:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x733a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.343736:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.343739:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343770:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007f130146/0xf62499ab92793626 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb0b expref: 49 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343776:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343780:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007f130146/0xf62499ab92793626 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb0b expref: 49 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343784:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.343791:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510788096:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.343793:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343800:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.343809:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c000 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.343813:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.343824:0:14245:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284812 00000100:00100000:3.0:1785220582.343830:0:14245:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfad2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+179:3652:x1871938510787968:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.343844:0:14245:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29502 count 1 00010000:00010000:3.0:1785220582.343856:0:14245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343867:0:14245:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001665ca7a/0xf62499ab9279365e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb43 expref: 49 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343874:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x733e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.343877:0:14245:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.343881:0:14245:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 4/0,1 mode: --/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343898:0:14245:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001665ca7a/0xf62499ab9279365e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb43 expref: 49 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343907:0:14245:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001665ca7a/0xf62499ab9279365e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb43 expref: 49 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.343916:0:14245:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510788224:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.343934:0:14245:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38cd80 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.343951:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.343955:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.343958:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+1375:3654:x1871938510784384:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14706us (15574us total) trans 4295070226 rc 0/0 00010000:00010000:3.0:1785220582.343963:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7200 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:2.0:1785220582.343965:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284800 00010000:00010000:3.0:1785220582.343966:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343974:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343984:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343986:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.343993:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.343994:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4e2fd5a/0xf62499ab927a5a66 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7310:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344001:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344007:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.344009:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.344010:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.344012:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.344019:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.344020:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.344027:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.344034:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.344041:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.344052:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.344169:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+180:3652:x1871938510784768:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14787us (15772us total) trans 4295070802 rc 0/0 00000100:00100000:2.0:1785220582.344175:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284801 00010000:00010000:2.0:1785220582.344193:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.344198:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344205:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.344209:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344221:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344232:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344239:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4476e42/0xf62499ab927a5a6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344253:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.344256:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344262:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.344265:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344273:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.344279:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd590c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.344282:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344288:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010cc6367/0xf62499ab927a5a74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344299:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344305:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.344307:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.344308:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344309:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.344314:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.344316:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344322:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344335:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.344406:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+1375:3652:x1871938510784640:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15330us (16048us total) trans 4295070227 rc 0/0 00010000:00010000:0.0:1785220582.344410:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220582.344413:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284799 00010000:00010000:0.0:1785220582.344414:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344421:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.344424:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344432:0:14222:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344432:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344441:0:14222:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344443:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344449:0:14222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344451:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001cb21e0b/0xf62499ab927a5a5f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7311:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344457:0:14222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344464:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.344466:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.344469:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344475:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.344477:0:14222:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1785220582.344478:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344485:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344494:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344500:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069a3eba5/0xf62499ab927a5a58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.344513:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.344514:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.344516:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.344535:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8ea00/2, rc 0, transno 4295070226 00000100:00100000:1.0:1785220582.344539:0:14222:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d998b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_068:lustre-MDT0000-mdtlov_UUID+180:3652:x1871938510784256:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15611us (16220us total) trans 4295070804 rc 0/0 00000100:00100000:0.0:1785220582.344543:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510784384:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.344546:0:14222:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284798 00000100:00100000:0.0:1785220582.344574:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510788480:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.344590:0:13867:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007bad18e6/0xf62499ab927933fd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8e2 expref: 79 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.344597:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.344601:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510786048:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.344617:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344625:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344635:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344644:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344651:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344662:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.344690:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.344695:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344703:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.344706:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344714:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344724:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344731:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000057f3a0f0/0xf62499ab927a5a82 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7312:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.344743:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.344745:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.344755:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.344761:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.344799:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d90ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1375:3654:x1871938510785920:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11639us (11880us total) trans 4295070228 rc 0/0 00000004:00080000:0.0:1785220582.344799:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90ba7b80/2, rc 0, transno 4295070228 00000100:00100000:1.0:1785220582.344805:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284804 00000100:00100000:0.0:1785220582.344807:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90ba7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510785920:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.344820:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510788480 00000100:00100000:1.0:1785220582.344825:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510788480 00000100:00100000:1.0:1785220582.344858:0:13867:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284813 00000100:00100000:1.0:1785220582.344865:0:13867:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfad3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1375:3654:x1871938510788480:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.344877:0:13867:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29464 count 1 00010000:00010000:1.0:1785220582.344890:0:13867:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344902:0:13867:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007c345a3b/0xf62499ab92793451 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f936 expref: 78 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.344909:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7318:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.344912:0:13867:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7318:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.344916:0:13867:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.344930:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:1.0:1785220582.344933:0:13867:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007c345a3b/0xf62499ab92793451 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f936 expref: 78 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.344937:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510788608:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.344943:0:13867:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007c345a3b/0xf62499ab92793451 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f936 expref: 78 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.344953:0:13867:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510788736:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.344959:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.344978:0:13867:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.344985:0:14205:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510788608 00000100:00100000:0.0:1785220582.344989:0:14205:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510788608 00000100:00100000:0.0:1785220582.345010:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284814 00000100:00100000:1.0:1785220582.345015:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d8ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510786816:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.345015:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfad1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1376:3654:x1871938510788608:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.345026:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29465 count 1 00010000:00010000:0.0:1785220582.345037:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.345039:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.345048:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009753e2bc/0xf62499ab9279345f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f944 expref: 78 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.345055:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510788352:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.345055:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7319:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.345058:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7319:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.345062:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.345071:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.345080:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009753e2bc/0xf62499ab9279345f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f944 expref: 78 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.345148:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510786816 00000100:00100000:2.0:1785220582.345152:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510786816 00000100:00100000:2.0:1785220582.345174:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510788352 00000100:00100000:2.0:1785220582.345177:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510788352 00000100:00100000:2.0:1785220582.345193:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284815 00000100:00100000:2.0:1785220582.345198:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8ae92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+181:3653:x1871938510786816:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.345209:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29499 count 1 00010000:00010000:2.0:1785220582.345221:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.345233:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002b3bd385/0xf62499ab92793634 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb19 expref: 49 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.345240:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x733b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.345243:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.345246:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 4/0,1 mode: --/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.345265:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002b3bd385/0xf62499ab92793634 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb19 expref: 49 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.345274:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002b3bd385/0xf62499ab92793634 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb19 expref: 49 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.345284:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0000_UUID:14176:1871938510788992:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.345303:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.345319:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284816 00000100:00100000:2.0:1785220582.345325:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8ae91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1377:3653:x1871938510788352:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.345337:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29463 count 1 00010000:00010000:2.0:1785220582.345348:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.345353:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+181:3654:x1871938510784512:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15750us (16926us total) trans 4295070803 rc 0/0 00010000:00010000:2.0:1785220582.345359:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2c973e8/0xf62499ab92793443 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f928 expref: 78 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.345360:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284802 00010000:00010000:2.0:1785220582.345365:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7317:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.345368:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7317:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.345372:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.345391:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c2c973e8/0xf62499ab92793443 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f928 expref: 78 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.345400:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c2c973e8/0xf62499ab92793443 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f928 expref: 78 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.345407:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510789120:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.345420:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9ae74c00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.345998:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009753e2bc/0xf62499ab9279345f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f944 expref: 78 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.346009:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510788864:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.346079:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.346143:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8df80/2, rc 0, transno 4295070803 00000100:00100000:0.0:1785220582.346159:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510784512:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.346182:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d955fdc00/2, rc 0, transno 4295070804 00000100:00100000:0.0:1785220582.346186:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784256:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.346202:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8e680/2, rc 0, transno 4295070802 00000100:00100000:0.0:1785220582.346205:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784768:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.346213:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8ed80/2, rc 0, transno 4295070227 00000100:00100000:0.0:1785220582.346220:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510784640:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.346263:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d985ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510789248:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.346292:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.346314:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510789248 00000100:00100000:0.0:1785220582.346319:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510789248 00000100:00100000:0.0:1785220582.346351:0:14169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284817 00000100:00100000:0.0:1785220582.346358:0:14169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1378:3652:x1871938510789248:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.346372:0:14169:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29466 count 1 00010000:00010000:0.0:1785220582.346386:0:14169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.346398:0:14169:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b3390773/0xf62499ab9279346d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f952 expref: 78 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.346406:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x731a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.346409:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.346413:0:14169:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.346432:0:14169:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b3390773/0xf62499ab9279346d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f952 expref: 78 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.346441:0:14169:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b3390773/0xf62499ab9279346d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f952 expref: 78 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.346452:0:14169:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98c84380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510789760:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.346477:0:14169:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.346489:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510789504:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.346507:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.346530:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1785220582.346535:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510789632:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.346539:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510789504 00000100:00100000:1.0:1785220582.346543:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510789504 00000100:00100000:0.0:1785220582.346556:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.346574:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510789632 00000100:00100000:1.0:1785220582.346577:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510789632 00000100:00100000:0.0:1785220582.346594:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.346596:0:14248:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284818 00000100:00100000:0.0:1785220582.346599:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510789376:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.346602:0:14248:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+182:3654:x1871938510789504:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.346614:0:14248:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29504 count 1 00010000:00010000:1.0:1785220582.346627:0:14248:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346638:0:14248:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002d3df8e1/0xf62499ab9279367a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb5f expref: 49 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346646:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7340:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.346649:0:14248:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7340:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.346653:0:14248:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346672:0:14248:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002d3df8e1/0xf62499ab9279367a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb5f expref: 49 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346682:0:14248:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002d3df8e1/0xf62499ab9279367a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb5f expref: 49 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.346690:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.346692:0:14248:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae93100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0000_UUID:14248:1871938510789888:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.346715:0:14248:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.346727:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510789376 00000100:00100000:1.0:1785220582.346730:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510789376 00000100:00100000:1.0:1785220582.346772:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284819 00000100:00100000:1.0:1785220582.346779:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+183:3654:x1871938510789632:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.346793:0:8441:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29505 count 1 00010000:00010000:1.0:1785220582.346804:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346815:0:8441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002f97cdaa/0xf62499ab92793688 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb6d expref: 49 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346827:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7341:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.346830:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7341:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.346833:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346852:0:8441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002f97cdaa/0xf62499ab92793688 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb6d expref: 49 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346860:0:8441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002f97cdaa/0xf62499ab92793688 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb6d expref: 49 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.346868:0:8441:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510790016:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.346891:0:8441:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.346907:0:14144:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284820 00000100:00100000:1.0:1785220582.346914:0:14144:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+183:3652:x1871938510789376:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.346926:0:14144:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29503 count 1 00010000:00010000:1.0:1785220582.346939:0:14144:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346951:0:14144:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000015fbdc30/0xf62499ab9279366c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb51 expref: 49 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346957:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x733f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.346959:0:14144:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.346963:0:14144:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346981:0:14144:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000015fbdc30/0xf62499ab9279366c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb51 expref: 49 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.346990:0:14144:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000015fbdc30/0xf62499ab9279366c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb51 expref: 49 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.347000:0:14144:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510790144:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.347022:0:14144:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.349976:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0001_UUID:14171:1871938510787840:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.350002:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510787712:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.350006:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.350016:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.350157:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0000_UUID:14176:1871938510788992:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.350171:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.350242:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0000_UUID:14156:1871938510788096:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.350256:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.351401:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510789120:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.351422:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.353377:0:14245:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_072:lustre-OST0000_UUID:14245:1871938510788224:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.353378:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510787584:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.353402:0:14245:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.353402:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.353484:0:14223:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfad3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0001_UUID:14223:1871938510787328:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.353503:0:14223:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.353534:0:13867:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:13867:1871938510788736:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.353549:0:13867:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.353575:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510787200:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.353585:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.353780:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.353817:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.353844:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.353861:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.353877:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.353896:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.353914:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.353938:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001536 00000100:00100000:2.0:1785220582.353944:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001536 00010000:00010000:2.0:1785220582.353981:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005ada6d94/0xf62499ab92793435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f91a expref: 79 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.354012:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001792 00000100:00100000:2.0:1785220582.354015:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001792 00010000:00010000:2.0:1785220582.354030:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb3ab728/0xf62499ab9279340b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f0 expref: 80 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.354049:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002048 00000100:00100000:2.0:1785220582.354052:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002048 00010000:00010000:2.0:1785220582.354064:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e0c30762/0xf62499ab92793419 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8fe expref: 81 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.354129:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002304 00000100:00100000:2.0:1785220582.354134:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002304 00010000:00010000:2.0:1785220582.354149:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001665ca7a/0xf62499ab9279365e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb43 expref: 50 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.354169:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002560 00000100:00100000:2.0:1785220582.354172:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002560 00010000:00010000:2.0:1785220582.354184:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002b3bd385/0xf62499ab92793634 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb19 expref: 51 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.354202:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001664 00000100:00100000:2.0:1785220582.354204:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001664 00010000:00010000:2.0:1785220582.354216:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cc2f525c/0xf62499ab92793650 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb35 expref: 52 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.354233:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520001920 00000100:00100000:2.0:1785220582.354236:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520001920 00010000:00010000:2.0:1785220582.354247:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005a28fa1f/0xf62499ab92793427 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f90c expref: 82 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.354265:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100738 00000100:00100000:2.0:1785220582.354274:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+82:2407:x1871938520001536:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.354288:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.354293:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005ada6d94/0xf62499ab92793435 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f91a expref: 82 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.354310:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005ada6d94/0xf62499ab92793435 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f91a expref: 82 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.354334:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005ada6d94/0xf62499ab92793435 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f91a expref: 82 pid: 14185 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.354350:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7316:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.354354:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7316:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.354359:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.354368:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.354389:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.354435:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+81:2407:x1871938520001536:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 162us (656us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.354443:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100738 00000100:00100000:2.0:1785220582.354460:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100739 00000100:00100000:3.0:1785220582.354472:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100740 00000100:00100000:3.0:1785220582.354481:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+80:2407:x1871938520002048:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.354489:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.354494:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e0c30762/0xf62499ab92793419 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8fe expref: 80 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354530:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e0c30762/0xf62499ab92793419 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8fe expref: 80 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.354537:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100741 00000100:00100000:0.0:1785220582.354545:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+52:2407:x1871938520002304:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.354549:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0c30762/0xf62499ab92793419 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8fe expref: 80 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354554:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.354559:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7314:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.354559:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001665ca7a/0xf62499ab9279365e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb43 expref: 52 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354562:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7314:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.354566:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354569:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001665ca7a/0xf62499ab9279365e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb43 expref: 52 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354575:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354582:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001665ca7a/0xf62499ab9279365e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb43 expref: 52 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354591:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.354592:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.354594:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.354598:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.354610:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+79:2407:x1871938520002048:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 132us (766us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.354617:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100740 00010000:00010000:0.0:1785220582.354619:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.354630:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100742 00000100:00100000:3.0:1785220582.354636:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+51:2407:x1871938520002560:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.354639:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.354644:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.354648:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002b3bd385/0xf62499ab92793634 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb19 expref: 51 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.354655:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+51:2407:x1871938520002304:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (795us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.354658:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002b3bd385/0xf62499ab92793634 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb19 expref: 51 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.354662:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100741 00000100:00100000:0.0:1785220582.354674:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100743 00010000:00010000:3.0:1785220582.354676:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b3bd385/0xf62499ab92793634 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb19 expref: 50 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.354680:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e13450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+50:2407:x1871938520001664:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.354686:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.354686:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.354689:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.354691:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cc2f525c/0xf62499ab92793650 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb35 expref: 49 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354694:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354703:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354707:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cc2f525c/0xf62499ab92793650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb35 expref: 49 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354718:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc2f525c/0xf62499ab92793650 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb35 expref: 49 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354720:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.354728:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.354731:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733d:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.354736:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+48:2407:x1871938520002560:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 101us (858us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.354736:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.354743:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100742 00010000:00010000:0.0:1785220582.354744:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.354767:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100744 00010000:00010000:0.0:1785220582.354773:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.354774:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+78:2407:x1871938520001920:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.354781:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.354785:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005a28fa1f/0xf62499ab92793427 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f90c expref: 78 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.354789:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e13450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2407:x1871938520001664:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 110us (892us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.354795:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100743 00010000:00010000:3.0:1785220582.354796:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005a28fa1f/0xf62499ab92793427 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f90c expref: 78 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354814:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a28fa1f/0xf62499ab92793427 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f90c expref: 78 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354814:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354823:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354826:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7315:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.354829:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7315:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.354832:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354840:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.354856:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.354895:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354905:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354928:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.354946:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+77:2407:x1871938520001920:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 172us (1030us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.354951:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100744 00010000:00010000:3.0:1785220582.354974:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354982:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354989:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.354995:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.355007:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.355013:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d8afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+184:3654:x1871938510787072:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12558us (13509us total) trans 4295070805 rc 0/0 00000100:00100000:0.0:1785220582.355021:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284809 00010000:00010000:0.0:1785220582.355041:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.355047:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355055:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.355059:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355067:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355079:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355103:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000be5f092/0xf62499ab927a5aa5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355119:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.355124:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355130:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.355133:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355139:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355149:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355156:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000041eeac35/0xf62499ab927a5a89 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7314:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355167:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.355169:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.355171:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.355181:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.355189:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355198:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355206:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355213:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355228:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.355322:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2407:x1871938520001792:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.355332:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.355338:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eb3ab728/0xf62499ab9279340b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f0 expref: 76 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.355353:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eb3ab728/0xf62499ab9279340b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f0 expref: 76 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.355370:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb3ab728/0xf62499ab9279340b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f8f0 expref: 76 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.355384:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7313:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.355387:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7313:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.355391:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.355399:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.355418:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.355527:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d8ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+1380:3654:x1871938510786560:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13185us (14057us total) trans 4295070230 rc 0/0 00000100:00100000:0.0:1785220582.355534:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284808 00010000:00010000:0.0:1785220582.355556:0:14245:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355563:0:14245:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355570:0:14245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355577:0:14245:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355590:0:14245:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.355650:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8ae92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+1379:3653:x1871938510786304:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14334us (15338us total) trans 4295070229 rc 0/0 00000100:00100000:3.0:1785220582.355657:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284805 00000100:00100000:0.0:1785220582.355721:0:14245:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfad2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_072:lustre-MDT0000-mdtlov_UUID+184:3652:x1871938510787968:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11891us (12159us total) trans 4295070806 rc 0/0 00000100:00100000:0.0:1785220582.355725:0:14245:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284812 00010000:00010000:0.0:1785220582.355740:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.355744:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355761:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.355764:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355775:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355787:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355794:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5a9e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7316:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355808:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.355811:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355817:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.355820:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355828:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355837:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 1/0,0 mode: --/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355844:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4d490c7/0xf62499ab927a5aba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.355858:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.355860:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.355861:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.355886:0:14223:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.355896:0:14223:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.355901:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8adc00/2, rc 0, transno 4295070230 00010000:00010000:1.0:1785220582.355904:0:14223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.355911:0:14223:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.355911:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510786560:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.355924:0:14223:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1785220582.355967:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8aea00/2, rc 0, transno 4295070805 00000100:00100000:0.0:1785220582.355977:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510787072:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.356000:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98c85f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510790272:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.356001:0:14223:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d8af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+1379:3654:x1871938510786432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14269us (15639us total) trans 4295070231 rc 0/0 00000100:00100000:1.0:1785220582.356007:0:14223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284806 00010000:00010000:1.0:1785220582.356023:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.356027:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.356032:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.356035:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.356038:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356047:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356058:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356065:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059f11396/0xf62499ab927a5a90 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.356071:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfad3100/2, rc 0, transno 4295070806 00000100:00100000:0.0:1785220582.356075:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfad3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510787968:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.356079:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.356081:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.356082:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.356124:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8aed80/2, rc 0, transno 4295070231 00000100:00100000:0.0:1785220582.356132:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510786432:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.356167:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98c84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510790400:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.356190:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.356214:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8dc00/2, rc 0, transno 4295070229 00000100:00100000:0.0:1785220582.356221:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510786304:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.356252:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.356259:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d98c84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510790528:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.356283:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.356344:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.356385:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510790272 00000100:00100000:1.0:1785220582.356390:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510790272 00000100:00100000:0.0:1785220582.356401:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.356416:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510790400 00000100:00100000:0.0:1785220582.356416:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002176 00000100:00100000:1.0:1785220582.356419:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510790400 00000100:00100000:0.0:1785220582.356419:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002176 00000100:00100000:1.0:1785220582.356433:0:14252:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510790528 00010000:00010000:0.0:1785220582.356434:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007f130146/0xf62499ab92793626 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb0b expref: 47 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.356435:0:14252:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510790528 00000100:00100000:0.0:1785220582.356449:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002432 00000100:00100000:1.0:1785220582.356452:0:14252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284821 00000100:00100000:0.0:1785220582.356452:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002432 00000100:00100000:1.0:1785220582.356458:0:14252:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+1381:3652:x1871938510790272:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.356461:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007c345a3b/0xf62499ab92793451 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f936 expref: 76 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.356475:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100745 00002000:00080000:1.0:1785220582.356479:0:14252:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29467 count 1 00000100:00100000:0.0:1785220582.356480:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c85c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2407:x1871938520002176:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.356487:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.356491:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007f130146/0xf62499ab92793626 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb0b expref: 47 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356496:0:14252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.356501:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007f130146/0xf62499ab92793626 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb0b expref: 47 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356508:0:14252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e1dfa9b1/0xf62499ab9279347b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f960 expref: 76 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.356513:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f130146/0xf62499ab92793626 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb0b expref: 47 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356516:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x731b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.356520:0:14252:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.356523:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.356524:0:14252:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.356526:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.356531:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.356538:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356545:0:14252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e1dfa9b1/0xf62499ab9279347b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f960 expref: 76 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.356552:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.356557:0:14252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e1dfa9b1/0xf62499ab9279347b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f960 expref: 76 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.356567:0:14252:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510790656:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.356572:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c85c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+46:2407:x1871938520002176:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (230us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.356578:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100745 00000100:00100000:0.0:1785220582.356591:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100746 00000100:00100000:0.0:1785220582.356597:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94d55c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2407:x1871938520002432:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1785220582.356601:0:14252:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd590c0 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.356604:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1785220582.356616:0:14175:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284822 00000100:00100000:1.0:1785220582.356623:0:14175:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+1381:3653:x1871938510790400:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.356634:0:14175:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29468 count 1 00010000:00010000:1.0:1785220582.356645:0:14175:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356657:0:14175:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000c12f599/0xf62499ab92793489 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f96e expref: 76 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356664:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x731c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.356667:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.356671:0:14175:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356688:0:14175:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000c12f599/0xf62499ab92793489 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f96e expref: 76 pid: 14145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356697:0:14175:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000c12f599/0xf62499ab92793489 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f96e expref: 76 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.356712:0:14175:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8ae93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510790784:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.356717:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+76:2407:x1871938520001792:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 2250us (2899us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.356725:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100739 00000100:00100000:1.0:1785220582.356729:0:14175:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8dd59300 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.356743:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284823 00000100:00100000:1.0:1785220582.356762:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+1381:3655:x1871938510790528:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.356774:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00080000:1.0:1785220582.356776:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29469 count 1 00010000:00010000:1.0:1785220582.356786:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356790:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356797:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d89ea711/0xf62499ab92793497 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f97c expref: 75 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356798:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356805:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x731d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.356809:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.356812:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356814:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356821:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356836:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.356905:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d89ea711/0xf62499ab92793497 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f97c expref: 75 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.356915:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d89ea711/0xf62499ab92793497 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f97c expref: 75 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.356925:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960ebfad3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510790912:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.356927:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfad0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+1382:3652:x1871938510786176:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14334us (15378us total) trans 4295070232 rc 0/0 00000100:00100000:2.0:1785220582.356933:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284810 00000100:00100000:1.0:1785220582.356945:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.356950:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.356954:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356961:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.356965:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356972:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356984:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.356991:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5aac lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7313:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.357000:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510791040:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.357007:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.357009:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.357011:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.357021:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.357023:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.357029:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.357037:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.357045:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.357056:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.357152:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8ae92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+184:3653:x1871938510786816:12345-0@lo:6:osp-syn-0-0.0 Request processed in 11952us (12111us total) trans 4295070807 rc 0/0 00000100:00100000:2.0:1785220582.357159:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284815 00000100:00100000:2.0:1785220582.357171:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510791040 00000100:00100000:2.0:1785220582.357175:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510791040 00000100:00100000:1.0:1785220582.357234:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d923a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510791168:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.357260:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1785220582.357352:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d90e8f800/2, rc 0, transno 4295070232 00000100:00100000:1.0:1785220582.357360:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d90e8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510786176:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.357360:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007c345a3b/0xf62499ab92793451 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f936 expref: 75 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357371:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007c345a3b/0xf62499ab92793451 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f936 expref: 75 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357385:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007c345a3b/0xf62499ab92793451 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f936 expref: 75 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357395:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7318:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.357398:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7318:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.357403:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.357416:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.357416:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357441:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.357444:0:13867:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.357453:0:13867:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.357455:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94d55c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+74:2407:x1871938520002432:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 859us (1054us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.357462:0:13867:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.357462:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100746 00010000:00010000:1.0:1785220582.357469:0:13867:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.357473:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002688 00000100:00100000:0.0:1785220582.357476:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002688 00010000:00010000:1.0:1785220582.357482:0:13867:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220582.357488:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c2c973e8/0xf62499ab92793443 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f928 expref: 74 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.357506:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100747 00000100:00100000:0.0:1785220582.357512:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a262680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+74:2407:x1871938520002688:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.357522:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.357526:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c2c973e8/0xf62499ab92793443 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f928 expref: 74 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357535:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c2c973e8/0xf62499ab92793443 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f928 expref: 74 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357548:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2c973e8/0xf62499ab92793443 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f928 expref: 74 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.357554:0:13867:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfad3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+1382:3654:x1871938510788480:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12690us (12956us total) trans 4295070233 rc 0/0 00000100:00100000:1.0:1785220582.357561:0:13867:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284813 00010000:00010000:0.0:1785220582.357563:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7317:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.357566:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7317:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.357570:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357577:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357600:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.357622:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a262680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+73:2407:x1871938520002688:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 110us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.357628:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100747 00010000:00010000:0.0:1785220582.357642:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.357646:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357654:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.357657:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357664:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357674:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 1/0,0 mode: --/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357685:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5acf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357699:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.357702:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357707:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee6a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.357710:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357716:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357724:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357800:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052d18d1d/0xf62499ab927a5ac1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7318:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13867 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.357813:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.357815:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.357816:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.357853:0:14176:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510791168 00000100:00100000:2.0:1785220582.357858:0:14176:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510791168 00000100:00100000:2.0:1785220582.357884:0:14176:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284824 00000100:00100000:2.0:1785220582.357890:0:14176:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+185:3655:x1871938510791040:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.357908:0:14176:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29506 count 1 00010000:00010000:2.0:1785220582.357924:0:14176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.357938:0:14176:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005b3cf899/0xf62499ab92793696 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb7b expref: 45 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.357945:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7342:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.357948:0:14176:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7342:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.357951:0:14176:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.357970:0:14176:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005b3cf899/0xf62499ab92793696 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb7b expref: 45 pid: 14247 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.357980:0:14176:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005b3cf899/0xf62499ab92793696 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb7b expref: 45 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.357992:0:14176:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0000_UUID:14176:1871938510791296:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.358019:0:14176:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.358033:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358041:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358049:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358057:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358069:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.358174:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8ae91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+1382:3653:x1871938510788352:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12848us (13100us total) trans 4295070234 rc 0/0 00000100:00100000:2.0:1785220582.358181:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284816 00000100:00100000:2.0:1785220582.358195:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284825 00000100:00100000:2.0:1785220582.358206:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d923a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+185:3652:x1871938510791168:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.358219:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29507 count 1 00010000:00010000:2.0:1785220582.358230:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358241:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001c1fb00a/0xf62499ab927936a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb89 expref: 45 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358249:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7343:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.358252:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7343:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.358256:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358279:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001c1fb00a/0xf62499ab927936a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb89 expref: 45 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358290:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001c1fb00a/0xf62499ab927936a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb89 expref: 45 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.358300:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0000_UUID:14258:1871938510791424:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.358317:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.358329:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.358333:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358340:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.358343:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358350:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358362:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358370:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5ad6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7317:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358384:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.358386:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.358387:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.358443:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.358490:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.358499:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002816 00000100:00100000:2.0:1785220582.358502:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002816 00010000:00010000:2.0:1785220582.358514:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000db74cbba/0xf62499ab92793642 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb27 expref: 46 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.358527:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003456 00000100:00100000:2.0:1785220582.358529:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003456 00010000:00010000:2.0:1785220582.358535:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000015fbdc30/0xf62499ab9279366c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb51 expref: 47 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.358545:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100748 00000100:00100000:2.0:1785220582.358549:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2407:x1871938520002816:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.358554:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.358557:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000db74cbba/0xf62499ab92793642 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb27 expref: 47 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358564:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000db74cbba/0xf62499ab92793642 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb27 expref: 47 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358574:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.358576:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.358579:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358584:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358591:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.358601:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b032d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2407:x1871938520002816:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 53us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.358616:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100748 00000100:00100000:2.0:1785220582.358624:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100749 00000100:00100000:2.0:1785220582.358628:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+46:2405:x1871938520003456:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.358633:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.358636:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000015fbdc30/0xf62499ab9279366c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb51 expref: 46 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358642:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000015fbdc30/0xf62499ab9279366c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb51 expref: 46 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358650:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.358651:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x733f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.358654:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358659:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.358665:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.358673:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b031880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+46:2405:x1871938520003456:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 45us (183us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.358676:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100749 00000100:00100000:2.0:1785220582.358699:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.358712:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.358722:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.358846:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.358873:0:8441:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d955fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8441:1871938510790016:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.358891:0:8441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.358916:0:14248:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae93100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_074:lustre-OST0000_UUID:14248:1871938510789888:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.358930:0:14248:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.358953:0:14144:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015fbdc30/0xf62499ab9279366c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb51 expref: 45 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.358965:0:14144:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d72fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:14144:1871938510790144:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.358975:0:14144:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.358983:0:14144:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.358991:0:14144:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.358999:0:14144:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359007:0:14144:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359035:0:14144:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.359187:0:14144:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+186:3652:x1871938510789376:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12272us (12494us total) trans 4295070808 rc 0/0 00000100:00100000:1.0:1785220582.359193:0:14144:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284820 00000100:00100000:1.0:1785220582.359215:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520002944 00000100:00100000:1.0:1785220582.359218:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520002944 00010000:00010000:1.0:1785220582.359234:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009753e2bc/0xf62499ab9279345f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f944 expref: 73 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359252:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003200 00000100:00100000:1.0:1785220582.359255:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003200 00010000:00010000:1.0:1785220582.359264:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002d3df8e1/0xf62499ab9279367a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb5f expref: 45 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359277:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003072 00000100:00100000:1.0:1785220582.359279:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003072 00010000:00010000:1.0:1785220582.359289:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b3390773/0xf62499ab9279346d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f952 expref: 74 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359304:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100750 00000100:00100000:1.0:1785220582.359310:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ad050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+74:2406:x1871938520002944:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.359319:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.359323:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009753e2bc/0xf62499ab9279345f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f944 expref: 74 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359338:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009753e2bc/0xf62499ab9279345f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f944 expref: 74 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359353:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7319:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.359356:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7319:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.359360:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359368:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359381:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.359389:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.359395:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ad050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+74:2406:x1871938520002944:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 86us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.359401:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100750 00010000:00010000:2.0:1785220582.359403:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.359408:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359415:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.359418:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359426:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359426:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003328 00000100:00100000:1.0:1785220582.359429:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003328 00010000:00010000:2.0:1785220582.359437:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359440:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002f97cdaa/0xf62499ab92793688 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb6d expref: 46 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359445:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5461ac7/0xf62499ab927a5af2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14144 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359458:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100751 00010000:00010000:2.0:1785220582.359460:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.359462:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.359464:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.359464:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+46:2406:x1871938520003200:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.359471:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220582.359474:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100752 00010000:00010000:1.0:1785220582.359475:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002d3df8e1/0xf62499ab9279367a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb5f expref: 46 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.359480:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+73:2406:x1871938520003072:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.359485:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002d3df8e1/0xf62499ab9279367a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb5f expref: 46 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359487:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.359492:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b3390773/0xf62499ab9279346d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f952 expref: 73 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359499:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d3df8e1/0xf62499ab9279367a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb5f expref: 46 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359507:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b3390773/0xf62499ab9279346d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f952 expref: 73 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359508:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7340:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.359511:0:14181:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7340:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.359515:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359517:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.359520:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.359523:0:14181:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359525:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359533:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359539:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.359542:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.359548:0:14248:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.359554:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+73:2406:x1871938520003072:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 75us (830us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220582.359557:0:14248:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.359560:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100752 00010000:00010000:1.0:1785220582.359565:0:14248:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.359572:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100753 00010000:00010000:1.0:1785220582.359572:0:14248:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.359578:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+45:2406:x1871938520003328:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.359585:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.359586:0:14248:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.359589:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002f97cdaa/0xf62499ab92793688 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb6d expref: 45 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359598:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002f97cdaa/0xf62499ab92793688 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb6d expref: 45 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359660:0:14248:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_074:lustre-MDT0000-mdtlov_UUID+186:3654:x1871938510789504:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13057us (13150us total) trans 4295070809 rc 0/0 00000100:00100000:1.0:1785220582.359666:0:14248:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284818 00010000:00010000:2.0:1785220582.359680:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f97cdaa/0xf62499ab92793688 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb6d expref: 45 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359691:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7341:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.359691:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b030700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+44:2406:x1871938520003200:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 227us (978us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.359695:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7341:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.359697:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100751 00010000:00010000:2.0:1785220582.359699:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359706:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359728:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1785220582.359728:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8ae90000/2, rc 0, transno 4295070809 00000100:00100000:1.0:1785220582.359738:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510789504:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.359742:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+43:2406:x1871938520003328:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 164us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.359761:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100753 00010000:00010000:1.0:1785220582.359761:0:8441:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359769:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359778:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359785:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.359795:0:8441:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.359803:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.359807:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359815:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.359818:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359864:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+186:3654:x1871938510789632:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13085us (13305us total) trans 4295070810 rc 0/0 00000100:00100000:1.0:1785220582.359873:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284819 00010000:00010000:2.0:1785220582.359893:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359908:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359914:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d42bdce/0xf62499ab927a5ae4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.359934:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfad2d80/2, rc 0, transno 4295070233 00000100:00100000:1.0:1785220582.359939:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfad2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510788480:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.359949:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8ae91880/2, rc 0, transno 4295070810 00010000:00010000:2.0:1785220582.359956:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220582.359956:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510789632:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.359959:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359966:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94ee7000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.359969:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.359977:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359979:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:2.0:1785220582.359987:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.359988:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a263480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510791680:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.359994:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc7c75e7/0xf62499ab927a5aeb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360007:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.360008:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.360011:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.360018:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.360060:0:14149:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510791680 00000100:00100000:2.0:1785220582.360064:0:14149:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510791680 00000004:00080000:1.0:1785220582.360071:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8ae93480/2, rc 0, transno 4295070808 00000100:00100000:1.0:1785220582.360079:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510789376:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.360124:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a261c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510791808:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.360149:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.360169:0:14166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510791808 00000100:00100000:1.0:1785220582.360172:0:14166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510791808 00000100:00100000:1.0:1785220582.360195:0:14166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284826 00000100:00100000:1.0:1785220582.360202:0:14166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b0e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+187:3655:x1871938510791680:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.360217:0:14166:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29509 count 1 00000100:00100000:2.0:1785220582.360231:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284827 00010000:00010000:1.0:1785220582.360234:0:14166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.360238:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a261880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+187:3655:x1871938510791808:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.360247:0:14166:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007fad1ac5/0xf62499ab927936c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fba5 expref: 42 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.360248:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29510 count 1 00010000:00010000:1.0:1785220582.360255:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7345:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.360258:0:14166:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7345:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.360259:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.360262:0:14166:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360269:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003be201d2/0xf62499ab927936ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbb3 expref: 42 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360276:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7346:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.360278:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7346:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.360281:0:14166:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007fad1ac5/0xf62499ab927936c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fba5 expref: 42 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360282:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.360293:0:14166:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007fad1ac5/0xf62499ab927936c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fba5 expref: 42 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360297:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003be201d2/0xf62499ab927936ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbb3 expref: 42 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.360305:0:14166:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a260a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510791936:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.360306:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003be201d2/0xf62499ab927936ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbb3 expref: 42 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.360315:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510792064:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.360327:0:14166:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.360329:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.360352:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220582.360359:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a262680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510791552:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.360376:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.360407:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510791552 00000100:00100000:2.0:1785220582.360412:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510791552 00000100:00100000:2.0:1785220582.360434:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284828 00000100:00100000:2.0:1785220582.360439:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a260700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+188:3652:x1871938510791552:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.360449:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29508 count 1 00010000:00010000:2.0:1785220582.360460:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360472:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000498038a1/0xf62499ab927936b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb97 expref: 42 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360479:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7344:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.360482:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7344:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.360486:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360503:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000498038a1/0xf62499ab927936b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb97 expref: 42 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.360513:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000498038a1/0xf62499ab927936b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb97 expref: 42 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.360523:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510792192:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.360543:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.360770:0:14205:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009753e2bc/0xf62499ab9279345f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f944 expref: 72 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.360788:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfad0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510788864:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.360808:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.360818:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.360825:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.360835:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.360842:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.360858:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.360977:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfad1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+1382:3654:x1871938510788608:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15961us (16016us total) trans 4295070235 rc 0/0 00000100:00100000:0.0:1785220582.360984:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284814 00010000:00010000:0.0:1785220582.361003:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.361007:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361019:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98cc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.361023:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361030:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361042:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361049:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd539a8f/0xf62499ab927a5ac8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7319:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361063:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.361065:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.361067:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.361073:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.361113:0:14169:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b3390773/0xf62499ab9279346d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f952 expref: 71 pid: 14250 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.361124:0:14169:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c84380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510789760:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.361136:0:14169:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361144:0:14169:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361152:0:14169:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361161:0:14169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361168:0:14169:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361182:0:14169:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.361254:0:14169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+1382:3652:x1871938510789248:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14895us (14961us total) trans 4295070236 rc 0/0 00000100:00100000:0.0:1785220582.361260:0:14169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284817 00010000:00010000:0.0:1785220582.361274:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.361278:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361284:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.361287:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361295:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361305:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361311:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003d360efc/0xf62499ab927a5add lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361325:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.361327:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.361329:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.361395:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94d54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510792448:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.361420:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.361442:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510792448 00000100:00100000:0.0:1785220582.361446:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510792448 00000100:00100000:0.0:1785220582.361469:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284829 00000100:00100000:0.0:1785220582.361474:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d94d55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+1382:3653:x1871938510792448:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.361488:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29471 count 1 00010000:00010000:0.0:1785220582.361501:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361512:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000077cbc2ad/0xf62499ab927934b3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f998 expref: 70 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361519:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x731f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.361522:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.361526:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361544:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000077cbc2ad/0xf62499ab927934b3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f998 expref: 70 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361554:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000077cbc2ad/0xf62499ab927934b3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f998 expref: 70 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.361564:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94d54a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510792576:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.361602:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.361636:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8ace00/2, rc 0, transno 4295070807 00000100:00100000:0.0:1785220582.361646:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510786816:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.361678:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.361685:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94d55c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510792704:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.361706:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1785220582.361736:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfad2680/2, rc 0, transno 4295070234 00000100:00100000:0.0:1785220582.361743:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfad2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510788352:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.361786:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906f9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510792832:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.361807:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.361828:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.361833:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d94d57100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510792320:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.361850:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.361867:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510792704 00000100:00100000:0.0:1785220582.361870:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510792704 00000100:00100000:0.0:1785220582.361890:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510792832 00000100:00100000:0.0:1785220582.361893:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510792832 00000100:00100000:0.0:1785220582.361904:0:14198:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510792320 00000100:00100000:0.0:1785220582.361906:0:14198:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510792320 00000100:00100000:0.0:1785220582.361920:0:14198:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284830 00000100:00100000:0.0:1785220582.361925:0:14198:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ebfad0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+189:3655:x1871938510792704:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.361936:0:14198:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29511 count 1 00010000:00010000:0.0:1785220582.361949:0:14198:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361960:0:14198:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000156ed14e/0xf62499ab927936dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc1 expref: 42 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361966:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7347:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.361969:0:14198:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7347:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.361972:0:14198:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361988:0:14198:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000156ed14e/0xf62499ab927936dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc1 expref: 42 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.361998:0:14198:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000156ed14e/0xf62499ab927936dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc1 expref: 42 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.362009:0:14198:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510792960:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.362032:0:14198:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.362048:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284831 00000100:00100000:0.0:1785220582.362055:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d906fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+1384:3655:x1871938510792832:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.362067:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29472 count 1 00010000:00010000:0.0:1785220582.362078:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362116:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000094d2bc9e/0xf62499ab927934c1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9a6 expref: 70 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362123:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7320:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.362127:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7320:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.362131:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362149:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000094d2bc9e/0xf62499ab927934c1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9a6 expref: 70 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362159:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000094d2bc9e/0xf62499ab927934c1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9a6 expref: 70 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.362170:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510793088:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.362196:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.362211:0:14261:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284832 00000100:00100000:0.0:1785220582.362218:0:14261:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d906fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+1384:3653:x1871938510792320:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.362229:0:14261:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29470 count 1 00010000:00010000:0.0:1785220582.362240:0:14261:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362252:0:14261:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000169dba9e/0xf62499ab927934a5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f98a expref: 70 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362265:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x731e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.362268:0:14261:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.362272:0:14261:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362290:0:14261:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000169dba9e/0xf62499ab927934a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f98a expref: 70 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.362300:0:14261:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000169dba9e/0xf62499ab927934a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f98a expref: 70 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.362311:0:14261:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510793216:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.362330:0:14261:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f300 going to sleep for 11 seconds 00000004:00080000:1.0:1785220582.362618:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960ebfad2300/2, rc 0, transno 4295070235 00000100:00100000:1.0:1785220582.362637:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfad2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510788608:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1785220582.362664:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d985ebb80/2, rc 0, transno 4295070236 00000100:00100000:1.0:1785220582.362668:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d985ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510789248:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.362779:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510793344:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.362809:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.362959:0:14221:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510793344 00000100:00100000:3.0:1785220582.362965:0:14221:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510793344 00000100:00100000:3.0:1785220582.362995:0:14221:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284833 00000100:00100000:3.0:1785220582.363002:0:14221:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a262300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+1385:3652:x1871938510793344:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.363016:0:14221:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29473 count 1 00010000:00010000:3.0:1785220582.363032:0:14221:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363048:0:14221:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ade06539/0xf62499ab927934cf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9b4 expref: 70 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363056:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7321:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.363059:0:14221:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7321:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.363063:0:14221:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363099:0:14221:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ade06539/0xf62499ab927934cf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9b4 expref: 70 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363109:0:14221:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ade06539/0xf62499ab927934cf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9b4 expref: 70 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.363126:0:14221:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510793472:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.363158:0:14221:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c000 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.363188:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363198:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363208:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363217:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363231:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.363385:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.363392:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.363404:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.363408:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.363415:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.363432:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.363439:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb25b45a/0xf62499ab927a5ab3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.363456:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.363457:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.363459:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.363531:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfad1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+190:3652:x1871938510786688:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19840us (20084us total) trans 4295070811 rc 0/0 00000100:00100000:3.0:1785220582.363539:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284811 00010000:00010000:3.0:1785220582.363578:0:8440:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db74cbba/0xf62499ab92793642 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb27 expref: 42 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.363591:0:8440:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b033100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8440:1871938510787456:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.363606:0:8440:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363620:0:8440:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363625:0:8440:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363633:0:8440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363640:0:8440:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.363653:0:8440:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.363739:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d8ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+190:3654:x1871938510786944:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21440us (22820us total) trans 4295070812 rc 0/0 00000100:00100000:3.0:1785220582.363746:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284807 00010000:00010000:0.0:1785220582.363820:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.363826:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.363834:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.363837:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.363846:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.363862:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.363868:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000142c7479/0xf62499ab927a5a97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.363883:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.363884:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.363886:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.363898:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510793600:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.363925:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.363946:0:14183:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510793600 00000100:00100000:0.0:1785220582.363949:0:14183:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510793600 00000100:00100000:0.0:1785220582.363975:0:14183:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284834 00000100:00100000:0.0:1785220582.363981:0:14183:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d906f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+1386:3653:x1871938510793600:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.363995:0:14183:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29474 count 1 00010000:00010000:0.0:1785220582.364009:0:14183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364022:0:14183:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f86764e3/0xf62499ab927934dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c2 expref: 70 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364029:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7322:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.364032:0:14183:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7322:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.364036:0:14183:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364058:0:14183:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f86764e3/0xf62499ab927934dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c2 expref: 70 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364068:0:14183:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f86764e3/0xf62499ab927934dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c2 expref: 70 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.364078:0:14183:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510793728:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.364152:0:14183:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000004:00080000:1.0:1785220582.364392:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8ad500/2, rc 0, transno 4295070811 00000100:00100000:1.0:1785220582.364442:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510786688:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.364584:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1785220582.364629:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510793856:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.364659:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.364739:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510793856 00000100:00100000:0.0:1785220582.364744:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510793856 00000100:00100000:0.0:1785220582.364817:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284835 00000100:00100000:0.0:1785220582.364826:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a263800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+190:3655:x1871938510793856:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.364843:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29512 count 1 00010000:00010000:0.0:1785220582.364859:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364872:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000014d44ab9/0xf62499ab927936ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbcf expref: 41 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364880:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7348:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.364883:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7348:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.364887:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364908:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000014d44ab9/0xf62499ab927936ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbcf expref: 41 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.364920:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000014d44ab9/0xf62499ab927936ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbcf expref: 41 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.364931:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d906faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510793984:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.364966:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.369184:0:14175:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae93b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510790784:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.369214:0:14175:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.370282:0:14252:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8ae90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_078:lustre-OST0001_UUID:14252:1871938510790656:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.370300:0:14252:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.373202:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.373278:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:2.0:1785220582.373297:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d8ac380/2, rc 0, transno 4295070812 00000100:00100000:0.0:1785220582.373304:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.373313:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d8ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510786944:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.373328:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.373357:0:14176:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b031f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_037:lustre-OST0000_UUID:14176:1871938510791296:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.373379:0:14176:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.373412:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b030a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0000_UUID:14258:1871938510791424:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.373426:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.373518:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003584 00000100:00100000:2.0:1785220582.373524:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003584 00000001:00080000:3.0:1785220582.373548:0:9520:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4295070236 is committed 00010000:00010000:2.0:1785220582.373556:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000c12f599/0xf62499ab92793489 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f96e expref: 71 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.373580:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003840 00000100:00100000:2.0:1785220582.373583:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003840 00010000:00010000:2.0:1785220582.373592:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e1dfa9b1/0xf62499ab9279347b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f960 expref: 72 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.373606:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004096 00000100:00100000:2.0:1785220582.373609:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004096 00010000:00010000:2.0:1785220582.373620:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001c1fb00a/0xf62499ab927936a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb89 expref: 42 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.373635:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004352 00000100:00100000:2.0:1785220582.373638:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004352 00010000:00010000:2.0:1785220582.373649:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003be201d2/0xf62499ab927936ce lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbb3 expref: 43 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.373669:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100754 00000100:00100000:2.0:1785220582.373678:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d906f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+72:2406:x1871938520003584:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.373693:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.373699:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000c12f599/0xf62499ab92793489 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f96e expref: 72 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373714:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000c12f599/0xf62499ab92793489 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f96e expref: 72 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373735:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000c12f599/0xf62499ab92793489 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f96e expref: 72 pid: 14145 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373761:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.373766:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.373772:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373782:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373811:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.373855:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d906f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+71:2406:x1871938520003584:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 177us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.373863:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100754 00000100:00100000:2.0:1785220582.373882:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100755 00000100:00100000:2.0:1785220582.373888:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d906fb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+70:2406:x1871938520003840:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.373896:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.373901:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e1dfa9b1/0xf62499ab9279347b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f960 expref: 70 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373915:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e1dfa9b1/0xf62499ab9279347b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f960 expref: 70 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373928:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e1dfa9b1/0xf62499ab9279347b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f960 expref: 70 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373937:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.373940:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731b:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.373940:0:14198:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_055:lustre-OST0000_UUID:14198:1871938510792960:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.373945:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373952:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.373956:0:14198:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.373969:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.373987:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d906fb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+69:2406:x1871938520003840:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (709us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.373993:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100755 00000100:00100000:2.0:1785220582.374006:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100756 00000100:00100000:2.0:1785220582.374012:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+43:2406:x1871938520004096:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.374014:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.374021:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.374026:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001c1fb00a/0xf62499ab927936a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb89 expref: 43 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374039:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001c1fb00a/0xf62499ab927936a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb89 expref: 43 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374052:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c1fb00a/0xf62499ab927936a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb89 expref: 43 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374062:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7343:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.374065:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7343:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.374070:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374077:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374126:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.374145:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+42:2406:x1871938520004096:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 133us (839us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.374151:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100756 00000100:00100000:2.0:1785220582.374162:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004480 00000100:00100000:2.0:1785220582.374165:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004480 00010000:00010000:2.0:1785220582.374178:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000156ed14e/0xf62499ab927936dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc1 expref: 42 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.374200:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100757 00000100:00100000:2.0:1785220582.374205:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce79c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+42:2404:x1871938520004352:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.374214:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.374218:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003be201d2/0xf62499ab927936ce lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbb3 expref: 42 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374228:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003be201d2/0xf62499ab927936ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbb3 expref: 42 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374241:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7346:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.374244:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7346:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.374248:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374255:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374266:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.374282:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce79c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+42:2404:x1871938520004352:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (953us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.374289:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100757 00000100:00100000:0.0:1785220582.374307:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.374654:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003712 00000100:00100000:2.0:1785220582.374660:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003712 00010000:00010000:2.0:1785220582.374675:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d89ea711/0xf62499ab92793497 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f97c expref: 69 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.374823:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960ebfad3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510790912:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.374841:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.374868:0:14166:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a260a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_028:lustre-OST0000_UUID:14166:1871938510791936:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.374879:0:14166:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.374934:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100758 00000100:00100000:2.0:1785220582.374941:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+41:2407:x1871938520004480:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.374950:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.374955:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000156ed14e/0xf62499ab927936dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc1 expref: 41 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374964:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000156ed14e/0xf62499ab927936dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc1 expref: 41 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374976:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000156ed14e/0xf62499ab927936dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc1 expref: 41 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374986:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7347:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.374988:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7347:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.374992:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.374999:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.375017:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.375036:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+40:2407:x1871938520004480:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 96us (1022us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.375042:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100758 00000100:00100000:2.0:1785220582.375053:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100759 00000100:00100000:2.0:1785220582.375058:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+69:2406:x1871938520003712:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.375065:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.375070:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d89ea711/0xf62499ab92793497 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f97c expref: 69 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.375079:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d89ea711/0xf62499ab92793497 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f97c expref: 69 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.375123:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d89ea711/0xf62499ab92793497 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f97c expref: 69 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.375134:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.375136:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.375140:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.375147:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.375167:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.375184:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+68:2406:x1871938520003712:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 125us (876us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.375189:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100759 00010000:00010000:1.0:1785220582.375300:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375309:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375318:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375326:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375342:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.375506:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+907:3655:x1871938510790528:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18756us (19222us total) trans 4295070237 rc 0/0 00000100:00100000:1.0:1785220582.375514:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284823 00010000:00010000:1.0:1785220582.375533:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.375539:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375547:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.375550:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375559:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375571:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375578:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5b07 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.375596:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.375597:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.375599:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.375620:0:3655:0:(client.c:3112:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4295070236 gen 2 00000004:00080000:1.0:1785220582.375626:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49d500, transno 4295068368 00000004:00080000:1.0:1785220582.375634:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49fb80, transno 4295068370 00000004:00080000:1.0:1785220582.375637:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b8380, transno 4295068372 00000004:00080000:1.0:1785220582.375639:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995f800, transno 4295068373 00000004:00080000:1.0:1785220582.375642:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979ea680, transno 4295068375 00000004:00080000:1.0:1785220582.375644:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979ead80, transno 4295068378 00000004:00080000:1.0:1785220582.375646:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bd180, transno 4295068379 00000004:00080000:1.0:1785220582.375649:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c61880, transno 4295068381 00000004:00080000:1.0:1785220582.375651:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c61500, transno 4295068384 00000004:00080000:1.0:1785220582.375654:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986ed80, transno 4295068385 00000004:00080000:1.0:1785220582.375657:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f69500, transno 4295068387 00000004:00080000:1.0:1785220582.375659:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986e680, transno 4295068391 00000004:00080000:1.0:1785220582.375661:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986c000, transno 4295068392 00000004:00080000:1.0:1785220582.375663:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c61c00, transno 4295068395 00000004:00080000:1.0:1785220582.375666:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c62d80, transno 4295068396 00000004:00080000:1.0:1785220582.375668:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c61180, transno 4295068398 00000004:00080000:1.0:1785220582.375671:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936be680, transno 4295068400 00000004:00080000:1.0:1785220582.375673:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c63b80, transno 4295068401 00000004:00080000:1.0:1785220582.375676:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995d500, transno 4295068402 00000004:00080000:1.0:1785220582.375678:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995d180, transno 4295068403 00000004:00080000:1.0:1785220582.375680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995dc00, transno 4295068404 00000004:00080000:1.0:1785220582.375683:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97957800, transno 4295068412 00000004:00080000:1.0:1785220582.375685:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97954000, transno 4295068413 00000004:00080000:1.0:1785220582.375687:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97954380, transno 4295068414 00000004:00080000:1.0:1785220582.375689:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97957480, transno 4295068415 00000004:00080000:1.0:1785220582.375691:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996d500, transno 4295068418 00000004:00080000:1.0:1785220582.375693:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996f100, transno 4295068419 00000004:00080000:1.0:1785220582.375695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19fb80, transno 4295068422 00000004:00080000:1.0:1785220582.375698:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19ed80, transno 4295068423 00000004:00080000:1.0:1785220582.375700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996ce00, transno 4295068424 00000004:00080000:1.0:1785220582.375703:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97955180, transno 4295068427 00000004:00080000:1.0:1785220582.375706:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6a680, transno 4295068429 00000004:00080000:1.0:1785220582.375708:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92368a80, transno 4295068431 00000004:00080000:1.0:1785220582.375711:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92369f80, transno 4295068433 00000004:00080000:1.0:1785220582.375713:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19e300, transno 4295068435 00000004:00080000:1.0:1785220582.375715:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5c000, transno 4295068437 00000004:00080000:1.0:1785220582.375718:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5d180, transno 4295068438 00000004:00080000:1.0:1785220582.375720:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5c700, transno 4295068440 00000004:00080000:1.0:1785220582.375723:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97954e00, transno 4295068441 00000004:00080000:1.0:1785220582.375725:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5f100, transno 4295068442 00000004:00080000:1.0:1785220582.375728:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5e680, transno 4295068443 00000004:00080000:1.0:1785220582.375730:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995ce00, transno 4295068447 00000004:00080000:1.0:1785220582.375732:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995ed80, transno 4295068448 00000004:00080000:1.0:1785220582.375734:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995f480, transno 4295068452 00000004:00080000:1.0:1785220582.375736:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19c000, transno 4295068453 00000004:00080000:1.0:1785220582.375738:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19f480, transno 4295068456 00000004:00080000:1.0:1785220582.375741:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5ea00, transno 4295068459 00000004:00080000:1.0:1785220582.375743:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e9500, transno 4295068460 00000004:00080000:1.0:1785220582.375745:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19f100, transno 4295068461 00000004:00080000:1.0:1785220582.375758:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995c380, transno 4295068462 00000004:00080000:1.0:1785220582.375760:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97955f80, transno 4295068465 00000004:00080000:1.0:1785220582.375763:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d7480, transno 4295068467 00000004:00080000:1.0:1785220582.375765:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97955880, transno 4295068470 00000004:00080000:1.0:1785220582.375767:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94def100, transno 4295068471 00000004:00080000:1.0:1785220582.375769:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94defb80, transno 4295068473 00000004:00080000:1.0:1785220582.375771:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dec000, transno 4295068474 00000004:00080000:1.0:1785220582.375773:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94def480, transno 4295068478 00000004:00080000:1.0:1785220582.375776:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dee680, transno 4295068479 00000004:00080000:1.0:1785220582.375779:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d6300, transno 4295068482 00000004:00080000:1.0:1785220582.375782:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5fb80, transno 4295068485 00000004:00080000:1.0:1785220582.375784:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5ed80, transno 4295068486 00000004:00080000:1.0:1785220582.375787:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f1180, transno 4295068491 00000004:00080000:1.0:1785220582.375789:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f3100, transno 4295068492 00000004:00080000:1.0:1785220582.375791:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f0000, transno 4295068493 00000004:00080000:1.0:1785220582.375794:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98413800, transno 4295068494 00000004:00080000:1.0:1785220582.375796:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00ca80, transno 4295068495 00000004:00080000:1.0:1785220582.375798:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6d180, transno 4295068497 00000004:00080000:1.0:1785220582.375801:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97443800, transno 4295068500 00000004:00080000:1.0:1785220582.375804:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97440a80, transno 4295068501 00000004:00080000:1.0:1785220582.375806:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97442300, transno 4295068504 00000004:00080000:1.0:1785220582.375808:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97441500, transno 4295068505 00000004:00080000:1.0:1785220582.375810:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94deca80, transno 4295068508 00000004:00080000:1.0:1785220582.375813:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dece00, transno 4295068511 00000004:00080000:1.0:1785220582.375815:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ded180, transno 4295068512 00000004:00080000:1.0:1785220582.375818:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8c700, transno 4295068515 00000004:00080000:1.0:1785220582.375820:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dedf80, transno 4295068516 00000004:00080000:1.0:1785220582.375822:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dedc00, transno 4295068517 00000004:00080000:1.0:1785220582.375824:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c28000, transno 4295068519 00000004:00080000:1.0:1785220582.375827:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c29880, transno 4295068520 00000004:00080000:1.0:1785220582.375830:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735dc00, transno 4295068523 00000004:00080000:1.0:1785220582.375832:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735f480, transno 4295068524 00000004:00080000:1.0:1785220582.375834:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735e680, transno 4295068525 00000004:00080000:1.0:1785220582.375836:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794ea00, transno 4295068528 00000004:00080000:1.0:1785220582.375839:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684e300, transno 4295068529 00000004:00080000:1.0:1785220582.375841:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986eed80, transno 4295068534 00000004:00080000:1.0:1785220582.375843:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794ed80, transno 4295068535 00000004:00080000:1.0:1785220582.375845:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae22300, transno 4295068537 00000004:00080000:1.0:1785220582.375848:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ec700, transno 4295068538 00000004:00080000:1.0:1785220582.375850:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae21880, transno 4295068541 00000004:00080000:1.0:1785220582.375853:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae21180, transno 4295068542 00000004:00080000:1.0:1785220582.375855:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae21c00, transno 4295068544 00000004:00080000:1.0:1785220582.375857:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ee680, transno 4295068548 00000004:00080000:1.0:1785220582.375859:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684ea00, transno 4295068551 00000004:00080000:1.0:1785220582.375862:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad4000, transno 4295068552 00000004:00080000:1.0:1785220582.375864:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad5f80, transno 4295068553 00000004:00080000:1.0:1785220582.375866:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad4e00, transno 4295068554 00000004:00080000:1.0:1785220582.375868:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbba300, transno 4295068555 00000004:00080000:1.0:1785220582.375871:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6c380, transno 4295068557 00000004:00080000:1.0:1785220582.375873:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de7800, transno 4295068558 00000004:00080000:1.0:1785220582.375876:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbe300, transno 4295068561 00000004:00080000:1.0:1785220582.375878:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa66680, transno 4295068563 00000004:00080000:1.0:1785220582.375881:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb9f80, transno 4295068567 00000004:00080000:1.0:1785220582.375883:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbbaa00, transno 4295068568 00000004:00080000:1.0:1785220582.375885:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa64000, transno 4295068570 00000004:00080000:1.0:1785220582.375887:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8d180, transno 4295068571 00000004:00080000:1.0:1785220582.375889:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa67480, transno 4295068573 00000004:00080000:1.0:1785220582.375891:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa67800, transno 4295068574 00000004:00080000:1.0:1785220582.375894:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b34ad80, transno 4295068579 00000004:00080000:1.0:1785220582.375896:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa64a80, transno 4295068583 00000004:00080000:1.0:1785220582.375898:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c29c00, transno 4295068584 00000004:00080000:1.0:1785220582.375900:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b349180, transno 4295068586 00000004:00080000:1.0:1785220582.375902:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c28380, transno 4295068587 00000004:00080000:1.0:1785220582.375904:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c2a300, transno 4295068588 00000004:00080000:1.0:1785220582.375906:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0dc380, transno 4295068589 00000004:00080000:1.0:1785220582.375908:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b349c00, transno 4295068592 00000004:00080000:1.0:1785220582.375910:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b348000, transno 4295068593 00000004:00080000:1.0:1785220582.375912:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ddf80, transno 4295068594 00000004:00080000:1.0:1785220582.375915:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b34b800, transno 4295068601 00000004:00080000:1.0:1785220582.375918:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96418000, transno 4295068602 00000004:00080000:1.0:1785220582.375920:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99376d80, transno 4295068603 00000004:00080000:1.0:1785220582.375922:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99374700, transno 4295068604 00000004:00080000:1.0:1785220582.375924:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ddc00, transno 4295068605 00000004:00080000:1.0:1785220582.375927:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e84700, transno 4295068608 00000004:00080000:1.0:1785220582.375930:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e87800, transno 4295068610 00000004:00080000:1.0:1785220582.375932:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ed500, transno 4295068614 00000004:00080000:1.0:1785220582.375934:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5efb80, transno 4295068617 00000004:00080000:1.0:1785220582.375936:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c549500, transno 4295068618 00000004:00080000:1.0:1785220582.375939:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b349f80, transno 4295068619 00000004:00080000:1.0:1785220582.375942:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae23800, transno 4295068620 00000004:00080000:1.0:1785220582.375944:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ed880, transno 4295068621 00000004:00080000:1.0:1785220582.375946:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae21500, transno 4295068628 00000004:00080000:1.0:1785220582.375948:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c0ad80, transno 4295068629 00000004:00080000:1.0:1785220582.375951:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99024a80, transno 4295068630 00000004:00080000:1.0:1785220582.375953:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b9aa00, transno 4295068631 00000004:00080000:1.0:1785220582.375955:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b98000, transno 4295068633 00000004:00080000:1.0:1785220582.375957:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b9ad80, transno 4295068634 00000004:00080000:1.0:1785220582.375960:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56f100, transno 4295068637 00000004:00080000:1.0:1785220582.375962:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b9b100, transno 4295068639 00000004:00080000:1.0:1785220582.375965:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91601c00, transno 4295068640 00000004:00080000:1.0:1785220582.375968:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e70380, transno 4295068645 00000004:00080000:1.0:1785220582.375970:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e72a00, transno 4295068646 00000004:00080000:1.0:1785220582.375973:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e71c00, transno 4295068647 00000004:00080000:1.0:1785220582.375975:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e73480, transno 4295068649 00000004:00080000:1.0:1785220582.375977:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fea00, transno 4295068650 00000004:00080000:1.0:1785220582.375980:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fc700, transno 4295068653 00000004:00080000:1.0:1785220582.375982:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fc380, transno 4295068654 00000004:00080000:1.0:1785220582.375985:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0ff100, transno 4295068658 00000004:00080000:1.0:1785220582.375987:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e72680, transno 4295068659 00000004:00080000:1.0:1785220582.375989:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fce00, transno 4295068660 00000004:00080000:1.0:1785220582.375991:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ed880, transno 4295068662 00000004:00080000:1.0:1785220582.375994:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fca80, transno 4295068663 00000004:00080000:1.0:1785220582.375996:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853f480, transno 4295068666 00000004:00080000:1.0:1785220582.375999:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853d880, transno 4295068668 00000004:00080000:1.0:1785220582.376001:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efa300, transno 4295068671 00000004:00080000:1.0:1785220582.376003:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99936680, transno 4295068674 00000004:00080000:1.0:1785220582.376005:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99934e00, transno 4295068675 00000004:00080000:1.0:1785220582.376008:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853e300, transno 4295068677 00000004:00080000:1.0:1785220582.376010:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853ca80, transno 4295068678 00000004:00080000:1.0:1785220582.376012:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efad80, transno 4295068680 00000004:00080000:1.0:1785220582.376014:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99936d80, transno 4295068681 00000004:00080000:1.0:1785220582.376017:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb79500, transno 4295068684 00000004:00080000:1.0:1785220582.376019:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b99500, transno 4295068685 00000004:00080000:1.0:1785220582.376022:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853fb80, transno 4295068688 00000004:00080000:1.0:1785220582.376024:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99024e00, transno 4295068689 00000004:00080000:1.0:1785220582.376026:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb79f80, transno 4295068694 00000004:00080000:1.0:1785220582.376028:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b99c00, transno 4295068695 00000004:00080000:1.0:1785220582.376030:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99934380, transno 4295068696 00000001:00080000:3.0:1785220582.376032:0:9520:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4295069367 is committed 00000004:00080000:1.0:1785220582.376032:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853ed80, transno 4295068697 00000004:00080000:1.0:1785220582.376034:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef8700, transno 4295068698 00000004:00080000:1.0:1785220582.376037:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a8380, transno 4295068699 00000004:00080000:1.0:1785220582.376039:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906ab480, transno 4295068701 00000004:00080000:1.0:1785220582.376042:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906ab100, transno 4295068703 00000004:00080000:1.0:1785220582.376044:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99027480, transno 4295068705 00000004:00080000:1.0:1785220582.376046:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e3480, transno 4295068707 00000004:00080000:1.0:1785220582.376048:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a8000, transno 4295068708 00000004:00080000:1.0:1785220582.376051:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ad8700, transno 4295068712 00000004:00080000:1.0:1785220582.376053:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e0e00, transno 4295068715 00000004:00080000:1.0:1785220582.376056:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef8e00, transno 4295068716 00000004:00080000:1.0:1785220582.376058:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d0000, transno 4295068718 00000004:00080000:1.0:1785220582.376061:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d3800, transno 4295068720 00000004:00080000:1.0:1785220582.376063:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d0700, transno 4295068721 00000004:00080000:1.0:1785220582.376065:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d0380, transno 4295068722 00000004:00080000:1.0:1785220582.376067:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef9180, transno 4295068725 00000004:00080000:1.0:1785220582.376069:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef1180, transno 4295068727 00000004:00080000:1.0:1785220582.376071:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef8380, transno 4295068728 00000004:00080000:1.0:1785220582.376074:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87ef8000, transno 4295068731 00000004:00080000:1.0:1785220582.376076:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1f800, transno 4295068735 00000004:00080000:1.0:1785220582.376078:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1e680, transno 4295068736 00000004:00080000:1.0:1785220582.376081:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2a680, transno 4295068737 00000004:00080000:1.0:1785220582.376083:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b29880, transno 4295068738 00000004:00080000:1.0:1785220582.376219:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1dc00, transno 4295068739 00000004:00080000:1.0:1785220582.376222:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1ca80, transno 4295068742 00000004:00080000:1.0:1785220582.376224:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2bb80, transno 4295068744 00000004:00080000:1.0:1785220582.376227:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ec000, transno 4295068749 00000004:00080000:1.0:1785220582.376229:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ef480, transno 4295068751 00000004:00080000:1.0:1785220582.376232:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964df480, transno 4295068752 00000004:00080000:1.0:1785220582.376234:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ee300, transno 4295068753 00000004:00080000:1.0:1785220582.376237:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0efb80, transno 4295068755 00000004:00080000:1.0:1785220582.376239:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0edc00, transno 4295068756 00000004:00080000:1.0:1785220582.376241:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bc380, transno 4295068759 00000004:00080000:1.0:1785220582.376244:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93570e00, transno 4295068760 00000004:00080000:1.0:1785220582.376246:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ef800, transno 4295068764 00000004:00080000:1.0:1785220582.376248:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dc380, transno 4295068765 00000004:00080000:1.0:1785220582.376251:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dfb80, transno 4295068767 00000004:00080000:1.0:1785220582.376253:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914d500, transno 4295068768 00000004:00080000:1.0:1785220582.376255:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964de680, transno 4295068769 00000004:00080000:1.0:1785220582.376258:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914e300, transno 4295068773 00000004:00080000:1.0:1785220582.376260:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b29500, transno 4295068776 00000004:00080000:1.0:1785220582.376262:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b28a80, transno 4295068779 00000004:00080000:1.0:1785220582.376265:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b29180, transno 4295068781 00000004:00080000:1.0:1785220582.376267:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2b100, transno 4295068782 00000004:00080000:1.0:1785220582.376269:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b29c00, transno 4295068783 00000004:00080000:1.0:1785220582.376272:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2a300, transno 4295068784 00000004:00080000:1.0:1785220582.376274:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b28380, transno 4295068790 00000004:00080000:1.0:1785220582.376276:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b28700, transno 4295068791 00000004:00080000:1.0:1785220582.376277:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bea00, transno 4295068793 00000004:00080000:1.0:1785220582.376280:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef5c00, transno 4295068794 00000004:00080000:1.0:1785220582.376282:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bf800, transno 4295068795 00000004:00080000:1.0:1785220582.376285:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef5180, transno 4295068796 00000004:00080000:1.0:1785220582.376287:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef4380, transno 4295068798 00000004:00080000:1.0:1785220582.376289:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91173b80, transno 4295068801 00000004:00080000:1.0:1785220582.376292:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c83480, transno 4295068802 00000004:00080000:1.0:1785220582.376294:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faab480, transno 4295068803 00000004:00080000:1.0:1785220582.376297:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c81f80, transno 4295068806 00000004:00080000:1.0:1785220582.376299:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87d47480, transno 4295068811 00000004:00080000:1.0:1785220582.376302:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc37b80, transno 4295068813 00000004:00080000:1.0:1785220582.376304:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb87480, transno 4295068814 00000004:00080000:1.0:1785220582.376307:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc34380, transno 4295068816 00000004:00080000:1.0:1785220582.376309:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc36d80, transno 4295068817 00000004:00080000:1.0:1785220582.376311:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb87100, transno 4295068818 00000004:00080000:1.0:1785220582.376314:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90405180, transno 4295068822 00000004:00080000:1.0:1785220582.376316:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91171180, transno 4295068824 00000004:00080000:1.0:1785220582.376319:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553a680, transno 4295068827 00000004:00080000:1.0:1785220582.376321:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa9c00, transno 4295068829 00000004:00080000:1.0:1785220582.376323:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa8e00, transno 4295068831 00000004:00080000:1.0:1785220582.376326:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faaad80, transno 4295068832 00000004:00080000:1.0:1785220582.376328:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90405f80, transno 4295068834 00000004:00080000:1.0:1785220582.376331:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af4380, transno 4295068836 00000004:00080000:1.0:1785220582.376333:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91172a00, transno 4295068837 00000004:00080000:1.0:1785220582.376335:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c169c00, transno 4295068841 00000004:00080000:1.0:1785220582.376337:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af6a00, transno 4295068845 00000004:00080000:1.0:1785220582.376340:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16b480, transno 4295068846 00000004:00080000:1.0:1785220582.376342:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16a680, transno 4295068847 00000004:00080000:1.0:1785220582.376344:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c19c00, transno 4295068849 00000004:00080000:1.0:1785220582.376347:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be2d80, transno 4295068851 00000004:00080000:1.0:1785220582.376349:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c168a80, transno 4295068852 00000004:00080000:1.0:1785220582.376351:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1a680, transno 4295068856 00000004:00080000:1.0:1785220582.376354:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1b100, transno 4295068858 00000004:00080000:1.0:1785220582.376356:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c18a80, transno 4295068859 00000004:00080000:1.0:1785220582.376358:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73a300, transno 4295068862 00000004:00080000:1.0:1785220582.376360:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f738700, transno 4295068863 00000004:00080000:1.0:1785220582.376363:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f739c00, transno 4295068864 00000004:00080000:1.0:1785220582.376365:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf9880, transno 4295068868 00000004:00080000:1.0:1785220582.376367:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f738e00, transno 4295068869 00000004:00080000:1.0:1785220582.376369:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b7800, transno 4295068872 00000004:00080000:1.0:1785220582.376372:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e0e00, transno 4295068875 00000004:00080000:1.0:1785220582.376374:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb2300, transno 4295068876 00000004:00080000:1.0:1785220582.376377:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa4380, transno 4295068877 00000004:00080000:1.0:1785220582.376379:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e1180, transno 4295068879 00000004:00080000:1.0:1785220582.376382:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e1f80, transno 4295068881 00000004:00080000:1.0:1785220582.376384:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb0000, transno 4295068883 00000004:00080000:1.0:1785220582.376386:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b39500, transno 4295068885 00000004:00080000:1.0:1785220582.376389:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bef480, transno 4295068887 00000004:00080000:1.0:1785220582.376390:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bef800, transno 4295068891 00000004:00080000:1.0:1785220582.376393:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93743800, transno 4295068892 00000004:00080000:1.0:1785220582.376395:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ceaa00, transno 4295068893 00000004:00080000:1.0:1785220582.376398:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa5500, transno 4295068894 00000004:00080000:1.0:1785220582.376400:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93743100, transno 4295068896 00000004:00080000:1.0:1785220582.376402:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91507480, transno 4295068899 00000004:00080000:1.0:1785220582.376404:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364b800, transno 4295068901 00000004:00080000:1.0:1785220582.376406:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93648a80, transno 4295068903 00000004:00080000:1.0:1785220582.376408:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb0a80, transno 4295068904 00000004:00080000:1.0:1785220582.376410:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb0700, transno 4295068905 00000004:00080000:1.0:1785220582.376412:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91506a00, transno 4295068906 00000004:00080000:1.0:1785220582.376414:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91504000, transno 4295068911 00000004:00080000:1.0:1785220582.376416:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93649880, transno 4295068912 00000004:00080000:1.0:1785220582.376418:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95450e00, transno 4295068914 00000004:00080000:1.0:1785220582.376421:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95451180, transno 4295068915 00000004:00080000:1.0:1785220582.376423:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95453800, transno 4295068917 00000004:00080000:1.0:1785220582.376426:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1c380, transno 4295068918 00000004:00080000:1.0:1785220582.376428:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93743b80, transno 4295068920 00000004:00080000:1.0:1785220582.376430:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1d180, transno 4295068921 00000004:00080000:1.0:1785220582.376432:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93629500, transno 4295068923 00000004:00080000:1.0:1785220582.376434:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93632a00, transno 4295068929 00000004:00080000:1.0:1785220582.376436:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93630e00, transno 4295068930 00000004:00080000:1.0:1785220582.376438:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93632300, transno 4295068931 00000004:00080000:1.0:1785220582.376440:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93649f80, transno 4295068934 00000004:00080000:1.0:1785220582.376443:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98484e00, transno 4295068935 00000004:00080000:1.0:1785220582.376445:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98485f80, transno 4295068936 00000004:00080000:1.0:1785220582.376448:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98487100, transno 4295068937 00000004:00080000:1.0:1785220582.376450:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cf800, transno 4295068938 00000004:00080000:1.0:1785220582.376452:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bedc00, transno 4295068941 00000004:00080000:1.0:1785220582.376455:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984ea00, transno 4295068942 00000004:00080000:1.0:1785220582.376457:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984df80, transno 4295068944 00000004:00080000:1.0:1785220582.376459:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984ca80, transno 4295068950 00000004:00080000:1.0:1785220582.376461:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bed180, transno 4295068951 00000004:00080000:1.0:1785220582.376463:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cea00, transno 4295068952 00000004:00080000:1.0:1785220582.376465:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b4700, transno 4295068953 00000004:00080000:1.0:1785220582.376468:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b17b80, transno 4295068955 00000004:00080000:1.0:1785220582.376470:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984d180, transno 4295068957 00000004:00080000:1.0:1785220582.376472:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef7100, transno 4295068961 00000004:00080000:1.0:1785220582.376474:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bee300, transno 4295068962 00000004:00080000:1.0:1785220582.376476:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef5880, transno 4295068963 00000004:00080000:1.0:1785220582.376479:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b5880, transno 4295068964 00000004:00080000:1.0:1785220582.376481:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef4a80, transno 4295068967 00000004:00080000:1.0:1785220582.376483:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1ce00, transno 4295068969 00000004:00080000:1.0:1785220582.376485:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bc000, transno 4295068971 00000004:00080000:1.0:1785220582.376488:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93570a80, transno 4295068972 00000004:00080000:1.0:1785220582.376490:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93573800, transno 4295068973 00000004:00080000:1.0:1785220582.376493:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96419f80, transno 4295068976 00000004:00080000:1.0:1785220582.376496:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc1c00, transno 4295068979 00000004:00080000:1.0:1785220582.376498:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e33800, transno 4295068980 00000004:00080000:1.0:1785220582.376500:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc0a80, transno 4295068981 00000004:00080000:1.0:1785220582.376502:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc0000, transno 4295068983 00000004:00080000:1.0:1785220582.376504:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc2d80, transno 4295068985 00000004:00080000:1.0:1785220582.376507:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e30a80, transno 4295068988 00000004:00080000:1.0:1785220582.376509:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986e300, transno 4295068989 00000004:00080000:1.0:1785220582.376512:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc1f80, transno 4295068990 00000004:00080000:1.0:1785220582.376514:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986ca80, transno 4295068992 00000004:00080000:1.0:1785220582.376516:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b38e00, transno 4295068993 00000004:00080000:1.0:1785220582.376518:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b7100, transno 4295068997 00000004:00080000:1.0:1785220582.376521:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c971c00, transno 4295068998 00000004:00080000:1.0:1785220582.376523:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbc380, transno 4295069000 00000004:00080000:1.0:1785220582.376526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b47b80, transno 4295069001 00000004:00080000:1.0:1785220582.376528:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98484000, transno 4295069005 00000004:00080000:1.0:1785220582.376530:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b44700, transno 4295069006 00000004:00080000:1.0:1785220582.376533:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ace00, transno 4295069007 00000004:00080000:1.0:1785220582.376535:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b44000, transno 4295069009 00000004:00080000:1.0:1785220582.376538:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b44e00, transno 4295069012 00000004:00080000:1.0:1785220582.376540:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98484700, transno 4295069014 00000004:00080000:1.0:1785220582.376542:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b46a00, transno 4295069016 00000004:00080000:1.0:1785220582.376544:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b45180, transno 4295069018 00000004:00080000:1.0:1785220582.376546:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98487480, transno 4295069019 00000004:00080000:1.0:1785220582.376549:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b47800, transno 4295069022 00000004:00080000:1.0:1785220582.376551:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98486d80, transno 4295069024 00000004:00080000:1.0:1785220582.376553:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98487b80, transno 4295069025 00000004:00080000:1.0:1785220582.376555:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b9880, transno 4295069026 00000004:00080000:1.0:1785220582.376558:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b8700, transno 4295069027 00000004:00080000:1.0:1785220582.376560:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c971180, transno 4295069029 00000004:00080000:1.0:1785220582.376562:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979bb480, transno 4295069037 00000004:00080000:1.0:1785220582.376564:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5d880, transno 4295069038 00000004:00080000:1.0:1785220582.376567:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c973b80, transno 4295069039 00000004:00080000:1.0:1785220582.376569:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6b800, transno 4295069041 00000004:00080000:1.0:1785220582.376571:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6b100, transno 4295069043 00000004:00080000:1.0:1785220582.376573:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f68700, transno 4295069044 00000004:00080000:1.0:1785220582.376576:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6a300, transno 4295069046 00000004:00080000:1.0:1785220582.376578:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f68380, transno 4295069049 00000004:00080000:1.0:1785220582.376580:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901af480, transno 4295069052 00000004:00080000:1.0:1785220582.376583:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6bb80, transno 4295069055 00000004:00080000:1.0:1785220582.376584:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901af100, transno 4295069056 00000004:00080000:1.0:1785220582.376586:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901aed80, transno 4295069057 00000004:00080000:1.0:1785220582.376588:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ac000, transno 4295069058 00000004:00080000:1.0:1785220582.376590:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901afb80, transno 4295069060 00000004:00080000:1.0:1785220582.376592:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ad500, transno 4295069063 00000004:00080000:1.0:1785220582.376595:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901adf80, transno 4295069066 00000004:00080000:1.0:1785220582.376597:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ef100, transno 4295069067 00000004:00080000:1.0:1785220582.376599:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ae300, transno 4295069068 00000004:00080000:1.0:1785220582.376601:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5c380, transno 4295069071 00000004:00080000:1.0:1785220582.376604:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dc700, transno 4295069073 00000004:00080000:1.0:1785220582.376606:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e84e00, transno 4295069075 00000004:00080000:1.0:1785220582.376608:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ad880, transno 4295069076 00000004:00080000:1.0:1785220582.376611:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a5180, transno 4295069079 00000004:00080000:1.0:1785220582.376613:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f55180, transno 4295069081 00000004:00080000:1.0:1785220582.376615:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964f4a80, transno 4295069082 00000004:00080000:1.0:1785220582.376617:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f57b80, transno 4295069085 00000004:00080000:1.0:1785220582.376620:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbdf80, transno 4295069088 00000004:00080000:1.0:1785220582.376623:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92164e00, transno 4295069091 00000004:00080000:1.0:1785220582.376625:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92167100, transno 4295069092 00000004:00080000:1.0:1785220582.376628:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe8a80, transno 4295069093 00000004:00080000:1.0:1785220582.376630:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfebb80, transno 4295069096 00000004:00080000:1.0:1785220582.376632:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe9880, transno 4295069097 00000004:00080000:1.0:1785220582.376634:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19dc00, transno 4295069099 00000004:00080000:1.0:1785220582.376637:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bdf80, transno 4295069102 00000004:00080000:1.0:1785220582.376639:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe8380, transno 4295069104 00000004:00080000:1.0:1785220582.376641:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfea300, transno 4295069105 00000004:00080000:1.0:1785220582.376643:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe8000, transno 4295069110 00000004:00080000:1.0:1785220582.376645:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bfb80, transno 4295069111 00000004:00080000:1.0:1785220582.376647:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98411500, transno 4295069114 00000004:00080000:1.0:1785220582.376649:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98413100, transno 4295069115 00000004:00080000:1.0:1785220582.376652:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98410a80, transno 4295069117 00000004:00080000:1.0:1785220582.376654:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98412a00, transno 4295069118 00000004:00080000:1.0:1785220582.376656:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98412300, transno 4295069119 00000004:00080000:1.0:1785220582.376658:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544aa00, transno 4295069121 00000004:00080000:1.0:1785220582.376661:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544b100, transno 4295069123 00000004:00080000:1.0:1785220582.376663:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544bb80, transno 4295069124 00000004:00080000:1.0:1785220582.376664:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95449880, transno 4295069130 00000004:00080000:1.0:1785220582.376667:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97956a00, transno 4295069131 00000004:00080000:1.0:1785220582.376669:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95449180, transno 4295069132 00000004:00080000:1.0:1785220582.376671:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95449c00, transno 4295069133 00000004:00080000:1.0:1785220582.376673:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544a300, transno 4295069136 00000004:00080000:1.0:1785220582.376675:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcabb80, transno 4295069138 00000004:00080000:1.0:1785220582.376678:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcaa300, transno 4295069139 00000004:00080000:1.0:1785220582.376680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6fb80, transno 4295069144 00000004:00080000:1.0:1785220582.376682:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca8380, transno 4295069145 00000004:00080000:1.0:1785220582.376684:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90407100, transno 4295069147 00000004:00080000:1.0:1785220582.376686:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90404700, transno 4295069148 00000004:00080000:1.0:1785220582.376688:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979a5180, transno 4295069151 00000004:00080000:1.0:1785220582.376691:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1f800, transno 4295069156 00000004:00080000:1.0:1785220582.376693:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1b800, transno 4295069157 00000004:00080000:1.0:1785220582.376695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1ca80, transno 4295069159 00000004:00080000:1.0:1785220582.376697:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1d500, transno 4295069161 00000004:00080000:1.0:1785220582.376700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1ed80, transno 4295069162 00000004:00080000:1.0:1785220582.376702:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c18e00, transno 4295069163 00000004:00080000:1.0:1785220582.376704:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1aa00, transno 4295069164 00000004:00080000:1.0:1785220582.376707:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af8000, transno 4295069168 00000004:00080000:1.0:1785220582.376709:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91506300, transno 4295069170 00000004:00080000:1.0:1785220582.376711:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91504700, transno 4295069176 00000004:00080000:1.0:1785220582.376714:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be0380, transno 4295069177 00000004:00080000:1.0:1785220582.376716:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc35880, transno 4295069178 00000004:00080000:1.0:1785220582.376719:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be0e00, transno 4295069180 00000004:00080000:1.0:1785220582.376721:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc36a00, transno 4295069181 00000004:00080000:1.0:1785220582.376724:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be1f80, transno 4295069183 00000004:00080000:1.0:1785220582.376726:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be3b80, transno 4295069185 00000004:00080000:1.0:1785220582.376729:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95423100, transno 4295069186 00000004:00080000:1.0:1785220582.376730:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95421180, transno 4295069187 00000004:00080000:1.0:1785220582.376733:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e3480, transno 4295069192 00000004:00080000:1.0:1785220582.376736:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91581180, transno 4295069193 00000004:00080000:1.0:1785220582.376738:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95420e00, transno 4295069196 00000004:00080000:1.0:1785220582.376741:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95420a80, transno 4295069198 00000004:00080000:1.0:1785220582.376743:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb86d80, transno 4295069199 00000004:00080000:1.0:1785220582.376755:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb84a80, transno 4295069201 00000004:00080000:1.0:1785220582.376759:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93570380, transno 4295069203 00000004:00080000:1.0:1785220582.376761:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93573b80, transno 4295069205 00000004:00080000:1.0:1785220582.376763:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00c380, transno 4295069207 00000004:00080000:1.0:1785220582.376766:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00d500, transno 4295069209 00000004:00080000:1.0:1785220582.376768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7fb80, transno 4295069210 00000004:00080000:1.0:1785220582.376770:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544b480, transno 4295069211 00000004:00080000:1.0:1785220582.376773:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb85c00, transno 4295069214 00000004:00080000:1.0:1785220582.376776:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e0380, transno 4295069217 00000004:00080000:1.0:1785220582.376778:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95422300, transno 4295069218 00000004:00080000:1.0:1785220582.376780:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95423b80, transno 4295069219 00000004:00080000:1.0:1785220582.376783:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb8e00, transno 4295069220 00000004:00080000:1.0:1785220582.376785:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbbb480, transno 4295069221 00000004:00080000:1.0:1785220582.376787:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e2300, transno 4295069225 00000004:00080000:1.0:1785220582.376789:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb8700, transno 4295069226 00000004:00080000:1.0:1785220582.376791:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1ea00, transno 4295069232 00000004:00080000:1.0:1785220582.376793:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f56680, transno 4295069233 00000004:00080000:1.0:1785220582.376796:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af5180, transno 4295069234 00000004:00080000:1.0:1785220582.376798:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa8000, transno 4295069235 00000004:00080000:1.0:1785220582.376800:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de6d80, transno 4295069238 00000004:00080000:1.0:1785220582.376803:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de5880, transno 4295069239 00000004:00080000:1.0:1785220582.376805:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f57100, transno 4295069241 00000004:00080000:1.0:1785220582.376808:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef6680, transno 4295069242 00000004:00080000:1.0:1785220582.376810:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853f100, transno 4295069246 00000004:00080000:1.0:1785220582.376812:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de6680, transno 4295069247 00000004:00080000:1.0:1785220582.376814:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875c380, transno 4295069253 00000004:00080000:1.0:1785220582.376817:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875e300, transno 4295069254 00000004:00080000:1.0:1785220582.376819:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e2680, transno 4295069257 00000004:00080000:1.0:1785220582.376822:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985eb100, transno 4295069259 00000004:00080000:1.0:1785220582.376825:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de5500, transno 4295069260 00000004:00080000:1.0:1785220582.376827:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e3b80, transno 4295069261 00000004:00080000:1.0:1785220582.376829:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de5f80, transno 4295069262 00000004:00080000:1.0:1785220582.376832:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99025880, transno 4295069263 00000004:00080000:1.0:1785220582.376834:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d63b800, transno 4295069266 00000004:00080000:1.0:1785220582.376836:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad9500, transno 4295069271 00000004:00080000:1.0:1785220582.376839:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a4000, transno 4295069272 00000004:00080000:1.0:1785220582.376841:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb87800, transno 4295069274 00000004:00080000:1.0:1785220582.376844:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a8380, transno 4295069275 00000004:00080000:1.0:1785220582.376846:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a4380, transno 4295069276 00000004:00080000:1.0:1785220582.376848:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad8000, transno 4295069278 00000004:00080000:1.0:1785220582.376851:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf8a80, transno 4295069280 00000004:00080000:1.0:1785220582.376853:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553a300, transno 4295069282 00000004:00080000:1.0:1785220582.376855:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d0380, transno 4295069285 00000004:00080000:1.0:1785220582.376857:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d0a80, transno 4295069288 00000004:00080000:1.0:1785220582.376859:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d1180, transno 4295069290 00000004:00080000:1.0:1785220582.376862:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d3b80, transno 4295069292 00000004:00080000:1.0:1785220582.376864:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d2680, transno 4295069293 00000004:00080000:1.0:1785220582.376866:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d0700, transno 4295069294 00000004:00080000:1.0:1785220582.376869:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d2a00, transno 4295069296 00000004:00080000:1.0:1785220582.376871:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d3800, transno 4295069297 00000004:00080000:1.0:1785220582.376873:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914ce00, transno 4295069304 00000004:00080000:1.0:1785220582.376875:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ae680, transno 4295069305 00000004:00080000:1.0:1785220582.376878:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de7b80, transno 4295069307 00000004:00080000:1.0:1785220582.376880:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e9180, transno 4295069308 00000004:00080000:1.0:1785220582.376882:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97588700, transno 4295069309 00000004:00080000:1.0:1785220582.376884:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853df80, transno 4295069311 00000004:00080000:1.0:1785220582.376887:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97589500, transno 4295069312 00000004:00080000:1.0:1785220582.376889:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb84700, transno 4295069314 00000004:00080000:1.0:1785220582.376892:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553aa00, transno 4295069316 00000004:00080000:1.0:1785220582.376894:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040f480, transno 4295069319 00000004:00080000:1.0:1785220582.376897:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e773b80, transno 4295069320 00000004:00080000:1.0:1785220582.376899:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86313100, transno 4295069321 00000004:00080000:1.0:1785220582.376901:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb35880, transno 4295069322 00000004:00080000:1.0:1785220582.376903:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e772680, transno 4295069323 00000004:00080000:1.0:1785220582.376905:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86311180, transno 4295069325 00000004:00080000:1.0:1785220582.376908:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb36a00, transno 4295069331 00000004:00080000:1.0:1785220582.376910:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa58380, transno 4295069332 00000004:00080000:1.0:1785220582.376913:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa59880, transno 4295069334 00000004:00080000:1.0:1785220582.376915:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5a680, transno 4295069335 00000004:00080000:1.0:1785220582.376918:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa59f80, transno 4295069338 00000004:00080000:1.0:1785220582.376920:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa59500, transno 4295069339 00000004:00080000:1.0:1785220582.376922:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5b480, transno 4295069341 00000004:00080000:1.0:1785220582.376924:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676ad80, transno 4295069344 00000004:00080000:1.0:1785220582.376927:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96769f80, transno 4295069346 00000004:00080000:1.0:1785220582.376929:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a1c00, transno 4295069347 00000004:00080000:1.0:1785220582.376931:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5ad80, transno 4295069351 00000004:00080000:1.0:1785220582.376933:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a3100, transno 4295069352 00000004:00080000:1.0:1785220582.376935:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b3800, transno 4295069354 00000004:00080000:1.0:1785220582.376938:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b3100, transno 4295069355 00000004:00080000:1.0:1785220582.376940:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6e300, transno 4295069358 00000004:00080000:1.0:1785220582.376943:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b2d80, transno 4295069359 00000004:00080000:1.0:1785220582.376945:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de4700, transno 4295069362 00000004:00080000:1.0:1785220582.376947:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9758a680, transno 4295069363 00000004:00080000:1.0:1785220582.376950:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92aa00, transno 4295069365 00000004:00080000:1.0:1785220582.376952:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c928e00, transno 4295069366 00000004:00080000:1.0:1785220582.376954:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de5180, transno 4295069368 00000004:00080000:1.0:1785220582.376956:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de6300, transno 4295069369 00000004:00080000:1.0:1785220582.376959:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92a300, transno 4295069370 00000004:00080000:1.0:1785220582.376961:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c929180, transno 4295069371 00000004:00080000:1.0:1785220582.376963:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6a680, transno 4295069372 00000004:00080000:1.0:1785220582.376965:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e771f80, transno 4295069373 00000004:00080000:1.0:1785220582.376968:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6aa00, transno 4295069374 00000004:00080000:1.0:1785220582.376970:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e770a80, transno 4295069375 00000004:00080000:1.0:1785220582.376972:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e770000, transno 4295069376 00000004:00080000:1.0:1785220582.376975:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb78e00, transno 4295069377 00000004:00080000:1.0:1785220582.376977:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e773800, transno 4295069378 00000004:00080000:1.0:1785220582.376979:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa59180, transno 4295069379 00000004:00080000:1.0:1785220582.376982:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b1f80, transno 4295069380 00000004:00080000:1.0:1785220582.376984:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b0380, transno 4295069381 00000004:00080000:1.0:1785220582.376987:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b1180, transno 4295069382 00000004:00080000:1.0:1785220582.376989:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a2300, transno 4295069383 00000004:00080000:1.0:1785220582.376992:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b0e00, transno 4295069384 00000004:00080000:1.0:1785220582.376994:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad0a80, transno 4295069385 00000004:00080000:1.0:1785220582.376996:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad0e00, transno 4295069386 00000004:00080000:1.0:1785220582.376998:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761880, transno 4295069387 00000004:00080000:1.0:1785220582.377001:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93762d80, transno 4295069388 00000004:00080000:1.0:1785220582.377003:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c928000, transno 4295069389 00000004:00080000:1.0:1785220582.377006:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859a680, transno 4295069390 00000004:00080000:1.0:1785220582.377008:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98598700, transno 4295069391 00000004:00080000:1.0:1785220582.377011:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98599880, transno 4295069392 00000004:00080000:1.0:1785220582.377013:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859b100, transno 4295069393 00000004:00080000:1.0:1785220582.377015:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859ad80, transno 4295069394 00000004:00080000:1.0:1785220582.377018:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9859b800, transno 4295069395 00000004:00080000:1.0:1785220582.377020:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5b800, transno 4295069396 00000004:00080000:1.0:1785220582.377022:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f58e00, transno 4295069397 00000004:00080000:1.0:1785220582.377024:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad0700, transno 4295069398 00000004:00080000:1.0:1785220582.377027:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5aa00, transno 4295069399 00000004:00080000:1.0:1785220582.377029:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5b480, transno 4295069400 00000004:00080000:1.0:1785220582.377031:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f58a80, transno 4295069401 00000004:00080000:1.0:1785220582.377033:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f59180, transno 4295069402 00000004:00080000:1.0:1785220582.377035:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f58380, transno 4295069403 00000004:00080000:1.0:1785220582.377037:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f58700, transno 4295069404 00000004:00080000:1.0:1785220582.377040:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a68380, transno 4295069405 00000004:00080000:1.0:1785220582.377042:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92ad80, transno 4295069406 00000004:00080000:1.0:1785220582.377044:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937b100, transno 4295069407 00000004:00080000:1.0:1785220582.377047:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92b800, transno 4295069408 00000004:00080000:1.0:1785220582.377049:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a69180, transno 4295069409 00000004:00080000:1.0:1785220582.377052:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c929f80, transno 4295069410 00000004:00080000:1.0:1785220582.377055:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99379c00, transno 4295069411 00000004:00080000:1.0:1785220582.377057:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99379500, transno 4295069412 00000004:00080000:1.0:1785220582.377059:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99378000, transno 4295069413 00000004:00080000:1.0:1785220582.377062:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5b800, transno 4295069414 00000004:00080000:1.0:1785220582.377064:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa58700, transno 4295069415 00000004:00080000:1.0:1785220582.377067:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6a680, transno 4295069416 00000004:00080000:1.0:1785220582.377069:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e772300, transno 4295069417 00000004:00080000:1.0:1785220582.377071:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98599c00, transno 4295069418 00000004:00080000:1.0:1785220582.377074:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a68a80, transno 4295069419 00000004:00080000:1.0:1785220582.377076:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a68e00, transno 4295069420 00000004:00080000:1.0:1785220582.377078:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6aa00, transno 4295069421 00000004:00080000:1.0:1785220582.377079:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faabb80, transno 4295069422 00000004:00080000:1.0:1785220582.377081:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c929880, transno 4295069423 00000004:00080000:1.0:1785220582.377195:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f5bb80, transno 4295069424 00000004:00080000:1.0:1785220582.377200:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96665f80, transno 4295069425 00000004:00080000:1.0:1785220582.377202:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f58000, transno 4295069426 00000004:00080000:1.0:1785220582.377204:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6ce00, transno 4295069427 00000004:00080000:1.0:1785220582.377206:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96664380, transno 4295069428 00000004:00080000:1.0:1785220582.377208:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96666680, transno 4295069429 00000004:00080000:1.0:1785220582.377211:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96665180, transno 4295069430 00000004:00080000:1.0:1785220582.377214:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96665c00, transno 4295069431 00000004:00080000:1.0:1785220582.377216:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96667100, transno 4295069432 00000004:00080000:1.0:1785220582.377218:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96666300, transno 4295069433 00000004:00080000:1.0:1785220582.377220:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96666d80, transno 4295069434 00000004:00080000:1.0:1785220582.377223:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96665880, transno 4295069435 00000004:00080000:1.0:1785220582.377225:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a68000, transno 4295069436 00000004:00080000:1.0:1785220582.377227:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a68700, transno 4295069437 00000004:00080000:1.0:1785220582.377229:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6b480, transno 4295069438 00000004:00080000:1.0:1785220582.377231:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a0e00, transno 4295069439 00000004:00080000:1.0:1785220582.377234:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9937a300, transno 4295069440 00000004:00080000:1.0:1785220582.377237:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a2a00, transno 4295069441 00000004:00080000:1.0:1785220582.377239:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99379f80, transno 4295069442 00000004:00080000:1.0:1785220582.377241:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a1180, transno 4295069443 00000004:00080000:1.0:1785220582.377244:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a2d80, transno 4295069444 00000004:00080000:1.0:1785220582.377247:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96667480, transno 4295069445 00000004:00080000:1.0:1785220582.377249:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a0700, transno 4295069446 00000004:00080000:1.0:1785220582.377251:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96664000, transno 4295069447 00000004:00080000:1.0:1785220582.377253:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96665500, transno 4295069448 00000004:00080000:1.0:1785220582.377255:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96664700, transno 4295069449 00000004:00080000:1.0:1785220582.377258:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92b480, transno 4295069450 00000004:00080000:1.0:1785220582.377260:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad7100, transno 4295069451 00000004:00080000:1.0:1785220582.377262:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dd180, transno 4295069452 00000004:00080000:1.0:1785220582.377265:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998df100, transno 4295069453 00000004:00080000:1.0:1785220582.377267:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95539180, transno 4295069454 00000004:00080000:1.0:1785220582.377269:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb34700, transno 4295069455 00000004:00080000:1.0:1785220582.377271:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dc380, transno 4295069456 00000004:00080000:1.0:1785220582.377273:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dea00, transno 4295069457 00000004:00080000:1.0:1785220582.377276:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa40380, transno 4295069458 00000004:00080000:1.0:1785220582.377278:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa41f80, transno 4295069459 00000004:00080000:1.0:1785220582.377281:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa43b80, transno 4295069460 00000004:00080000:1.0:1785220582.377283:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa40e00, transno 4295069461 00000004:00080000:1.0:1785220582.377285:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa42300, transno 4295069462 00000004:00080000:1.0:1785220582.377287:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa42d80, transno 4295069463 00000004:00080000:1.0:1785220582.377289:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa40a80, transno 4295069464 00000004:00080000:1.0:1785220582.377291:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98598a80, transno 4295069465 00000004:00080000:1.0:1785220582.377293:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa41500, transno 4295069466 00000004:00080000:1.0:1785220582.377295:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98599500, transno 4295069467 00000004:00080000:1.0:1785220582.377297:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa40000, transno 4295069468 00000004:00080000:1.0:1785220582.377299:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa43480, transno 4295069469 00000004:00080000:1.0:1785220582.377302:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa43800, transno 4295069470 00000004:00080000:1.0:1785220582.377304:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98580e00, transno 4295069471 00000004:00080000:1.0:1785220582.377306:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93763480, transno 4295069472 00000004:00080000:1.0:1785220582.377309:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98580000, transno 4295069473 00000004:00080000:1.0:1785220582.377311:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93762300, transno 4295069474 00000004:00080000:1.0:1785220582.377313:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59ed80, transno 4295069475 00000004:00080000:1.0:1785220582.377315:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90409f80, transno 4295069476 00000004:00080000:1.0:1785220582.377318:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90408a80, transno 4295069477 00000004:00080000:1.0:1785220582.377320:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59d500, transno 4295069478 00000004:00080000:1.0:1785220582.377322:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90408700, transno 4295069479 00000004:00080000:1.0:1785220582.377325:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59c000, transno 4295069480 00000004:00080000:1.0:1785220582.377327:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040a300, transno 4295069481 00000004:00080000:1.0:1785220582.377329:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59ca80, transno 4295069482 00000004:00080000:1.0:1785220582.377331:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f59880, transno 4295069483 00000004:00080000:1.0:1785220582.377333:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f59c00, transno 4295069484 00000004:00080000:1.0:1785220582.377335:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c0a80, transno 4295069485 00000004:00080000:1.0:1785220582.377337:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c2300, transno 4295069486 00000004:00080000:1.0:1785220582.377339:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c1f80, transno 4295069487 00000004:00080000:1.0:1785220582.377342:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93763100, transno 4295069488 00000004:00080000:1.0:1785220582.377344:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c1500, transno 4295069489 00000004:00080000:1.0:1785220582.377347:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c0380, transno 4295069490 00000004:00080000:1.0:1785220582.377349:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae90e00, transno 4295069491 00000004:00080000:1.0:1785220582.377352:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fdc00, transno 4295069492 00000004:00080000:1.0:1785220582.377354:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae91500, transno 4295069493 00000004:00080000:1.0:1785220582.377357:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b58e00, transno 4295069494 00000004:00080000:1.0:1785220582.377359:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b59c00, transno 4295069495 00000004:00080000:1.0:1785220582.377361:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae90700, transno 4295069496 00000004:00080000:1.0:1785220582.377363:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae93800, transno 4295069497 00000004:00080000:1.0:1785220582.377365:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae92d80, transno 4295069498 00000004:00080000:1.0:1785220582.377368:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fd880, transno 4295069499 00000004:00080000:1.0:1785220582.377371:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fc700, transno 4295069500 00000004:00080000:1.0:1785220582.377373:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936ff800, transno 4295069501 00000004:00080000:1.0:1785220582.377376:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fdf80, transno 4295069502 00000004:00080000:1.0:1785220582.377378:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fd500, transno 4295069503 00000004:00080000:1.0:1785220582.377380:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936ff100, transno 4295069504 00000004:00080000:1.0:1785220582.377382:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fed80, transno 4295069505 00000004:00080000:1.0:1785220582.377384:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fd180, transno 4295069506 00000004:00080000:1.0:1785220582.377386:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fc380, transno 4295069507 00000004:00080000:1.0:1785220582.377388:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936ff480, transno 4295069508 00000004:00080000:1.0:1785220582.377390:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a69500, transno 4295069509 00000004:00080000:1.0:1785220582.377392:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa42680, transno 4295069510 00000004:00080000:1.0:1785220582.377395:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fce00, transno 4295069511 00000004:00080000:1.0:1785220582.377397:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae90380, transno 4295069512 00000004:00080000:1.0:1785220582.377400:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae91c00, transno 4295069513 00000004:00080000:1.0:1785220582.377402:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa41180, transno 4295069514 00000004:00080000:1.0:1785220582.377404:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fe680, transno 4295069515 00000004:00080000:1.0:1785220582.377406:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93638000, transno 4295069516 00000004:00080000:1.0:1785220582.377408:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93639c00, transno 4295069517 00000004:00080000:1.0:1785220582.377411:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae91f80, transno 4295069518 00000004:00080000:1.0:1785220582.377413:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fc000, transno 4295069519 00000004:00080000:1.0:1785220582.377415:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5b800, transno 4295069520 00000004:00080000:1.0:1785220582.377418:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b58700, transno 4295069521 00000004:00080000:1.0:1785220582.377420:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b59f80, transno 4295069522 00000004:00080000:1.0:1785220582.377422:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b58380, transno 4295069523 00000004:00080000:1.0:1785220582.377424:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5bb80, transno 4295069524 00000004:00080000:1.0:1785220582.377426:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b58000, transno 4295069525 00000004:00080000:1.0:1785220582.377429:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb35f80, transno 4295069526 00000004:00080000:1.0:1785220582.377431:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dd880, transno 4295069527 00000004:00080000:1.0:1785220582.377433:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6b800, transno 4295069528 00000004:00080000:1.0:1785220582.377435:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb36300, transno 4295069529 00000004:00080000:1.0:1785220582.377438:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb37480, transno 4295069530 00000004:00080000:1.0:1785220582.377440:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dce00, transno 4295069531 00000004:00080000:1.0:1785220582.377442:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dd500, transno 4295069532 00000004:00080000:1.0:1785220582.377445:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dfb80, transno 4295069533 00000004:00080000:1.0:1785220582.377447:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998df800, transno 4295069534 00000004:00080000:1.0:1785220582.377450:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96664a80, transno 4295069535 00000004:00080000:1.0:1785220582.377452:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761c00, transno 4295069536 00000004:00080000:1.0:1785220582.377454:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99270000, transno 4295069537 00000004:00080000:1.0:1785220582.377457:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e773480, transno 4295069538 00000004:00080000:1.0:1785220582.377459:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dc000, transno 4295069539 00000004:00080000:1.0:1785220582.377461:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81219500, transno 4295069540 00010000:00010000:3.0:1785220582.377463:0:14175:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.377464:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f8a80, transno 4295069541 00000004:00080000:1.0:1785220582.377466:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c683480, transno 4295069542 00000004:00080000:1.0:1785220582.377468:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c680380, transno 4295069543 00000004:00080000:1.0:1785220582.377470:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040b480, transno 4295069544 00000004:00080000:1.0:1785220582.377473:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a1500, transno 4295069545 00000004:00080000:1.0:1785220582.377476:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a0a80, transno 4295069546 00000004:00080000:1.0:1785220582.377478:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b59500, transno 4295069547 00000004:00080000:1.0:1785220582.377481:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a3b80, transno 4295069548 00010000:00010000:3.0:1785220582.377482:0:14175:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.377483:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b59180, transno 4295069549 00000004:00080000:1.0:1785220582.377485:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81218000, transno 4295069550 00000004:00080000:1.0:1785220582.377488:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a3480, transno 4295069551 00000004:00080000:1.0:1785220582.377490:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c680700, transno 4295069552 00000004:00080000:1.0:1785220582.377492:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81219180, transno 4295069553 00010000:00010000:3.0:1785220582.377493:0:14175:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.377494:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93639500, transno 4295069554 00000004:00080000:1.0:1785220582.377497:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa41c00, transno 4295069555 00000004:00080000:1.0:1785220582.377499:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99272680, transno 4295069556 00010000:00010000:3.0:1785220582.377501:0:14175:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.377501:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad1c00, transno 4295069557 00000004:00080000:1.0:1785220582.377504:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dc700, transno 4295069558 00000004:00080000:1.0:1785220582.377506:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c2d80, transno 4295069559 00000004:00080000:1.0:1785220582.377509:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99270700, transno 4295069560 00000004:00080000:1.0:1785220582.377511:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d310380, transno 4295069561 00000004:00080000:1.0:1785220582.377513:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d310e00, transno 4295069562 00000004:00080000:1.0:1785220582.377515:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d313b80, transno 4295069563 00000004:00080000:1.0:1785220582.377517:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93639880, transno 4295069564 00000004:00080000:1.0:1785220582.377520:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936ffb80, transno 4295069565 00000004:00080000:1.0:1785220582.377522:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5a300, transno 4295069566 00000004:00080000:1.0:1785220582.377524:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936fe300, transno 4295069567 00010000:00010000:3.0:1785220582.377526:0:14175:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1785220582.377526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95b5b100, transno 4295069568 00000004:00080000:1.0:1785220582.377529:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59e680, transno 4295069569 00000004:00080000:1.0:1785220582.377531:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ae92a00, transno 4295069570 00000004:00080000:1.0:1785220582.377533:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90408e00, transno 4295069571 00000004:00080000:1.0:1785220582.377535:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9363a300, transno 4295069572 00000004:00080000:1.0:1785220582.377537:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93638380, transno 4295069573 00000004:00080000:1.0:1785220582.377540:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90409880, transno 4295069574 00000004:00080000:1.0:1785220582.377542:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c59f800, transno 4295069575 00000004:00080000:1.0:1785220582.377544:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae68e00, transno 4295069576 00000004:00080000:1.0:1785220582.377546:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040b800, transno 4295069577 00000004:00080000:1.0:1785220582.377548:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93638e00, transno 4295069578 00000004:00080000:1.0:1785220582.377550:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90a6a300, transno 4295069579 00000004:00080000:1.0:1785220582.377553:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99270380, transno 4295069580 00000004:00080000:1.0:1785220582.377555:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d311500, transno 4295069581 00000004:00080000:1.0:1785220582.377557:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d310a80, transno 4295069582 00000004:00080000:1.0:1785220582.377560:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d56680, transno 4295069583 00000004:00080000:1.0:1785220582.377562:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d54e00, transno 4295069584 00000004:00080000:1.0:1785220582.377564:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f8a80, transno 4295069585 00000004:00080000:1.0:1785220582.377566:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d54000, transno 4295069586 00000004:00080000:1.0:1785220582.377568:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d313100, transno 4295069587 00000004:00080000:1.0:1785220582.377571:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f8700, transno 4295069588 00000004:00080000:1.0:1785220582.377573:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998de300, transno 4295069589 00000004:00080000:1.0:1785220582.377575:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81219880, transno 4295069590 00000004:00080000:1.0:1785220582.377577:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998ded80, transno 4295069591 00000004:00080000:1.0:1785220582.377580:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998dca80, transno 4295069592 00000004:00080000:1.0:1785220582.377582:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121ad80, transno 4295069593 00000004:00080000:1.0:1785220582.377584:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa41880, transno 4295069594 00000004:00080000:1.0:1785220582.377587:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add6d80, transno 4295069595 00000004:00080000:1.0:1785220582.377590:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e81218380, transno 4295069596 00000004:00080000:1.0:1785220582.377592:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add4a80, transno 4295069597 00000004:00080000:1.0:1785220582.377594:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add7800, transno 4295069598 00000004:00080000:1.0:1785220582.377597:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a707100, transno 4295069599 00000004:00080000:1.0:1785220582.377599:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a707480, transno 4295069600 00000004:00080000:1.0:1785220582.377602:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f3b80, transno 4295069601 00000004:00080000:1.0:1785220582.377604:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d57b80, transno 4295069602 00000004:00080000:1.0:1785220582.377607:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a705f80, transno 4295069603 00000004:00080000:1.0:1785220582.377609:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add7b80, transno 4295069604 00000004:00080000:1.0:1785220582.377611:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a706300, transno 4295069605 00000004:00080000:1.0:1785220582.377614:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add6300, transno 4295069606 00000004:00080000:1.0:1785220582.377616:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d56300, transno 4295069607 00000004:00080000:1.0:1785220582.377618:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae70e00, transno 4295069608 00000004:00080000:1.0:1785220582.377620:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d57800, transno 4295069609 00000004:00080000:1.0:1785220582.377622:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d56d80, transno 4295069610 00000004:00080000:1.0:1785220582.377626:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97986d80, transno 4295069611 00000004:00080000:1.0:1785220582.377628:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97987100, transno 4295069612 00000004:00080000:1.0:1785220582.377630:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d55180, transno 4295069613 00000004:00080000:1.0:1785220582.377632:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae71880, transno 4295069614 00000004:00080000:1.0:1785220582.377634:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d55500, transno 4295069615 00000004:00080000:1.0:1785220582.377636:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d55880, transno 4295069616 00000004:00080000:1.0:1785220582.377638:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d56a00, transno 4295069617 00000004:00080000:1.0:1785220582.377640:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d699180, transno 4295069618 00000004:00080000:1.0:1785220582.377643:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d57480, transno 4295069619 00000004:00080000:1.0:1785220582.377645:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d698380, transno 4295069620 00000004:00080000:1.0:1785220582.377648:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d698000, transno 4295069621 00000004:00080000:1.0:1785220582.377650:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69b100, transno 4295069622 00000004:00080000:1.0:1785220582.377652:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69a300, transno 4295069623 00000004:00080000:1.0:1785220582.377655:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d698a80, transno 4295069624 00000004:00080000:1.0:1785220582.377658:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d69b800, transno 4295069625 00000004:00080000:1.0:1785220582.377660:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92bb80, transno 4295069626 00000004:00080000:1.0:1785220582.377663:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9363b480, transno 4295069627 00000004:00080000:1.0:1785220582.377665:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8d500, transno 4295069628 00000004:00080000:1.0:1785220582.377667:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94938380, transno 4295069629 00000004:00080000:1.0:1785220582.377669:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8fb80, transno 4295069630 00000004:00080000:1.0:1785220582.377671:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8d180, transno 4295069631 00000004:00080000:1.0:1785220582.377673:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8e300, transno 4295069632 00000004:00080000:1.0:1785220582.377676:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8ce00, transno 4295069633 00000004:00080000:1.0:1785220582.377679:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94939500, transno 4295069634 00000004:00080000:1.0:1785220582.377682:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493b800, transno 4295069635 00000004:00080000:1.0:1785220582.377684:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97984380, transno 4295069636 00000004:00080000:1.0:1785220582.377686:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97985500, transno 4295069637 00000004:00080000:1.0:1785220582.377688:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8f100, transno 4295069638 00000004:00080000:1.0:1785220582.377690:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97987800, transno 4295069639 00000004:00080000:1.0:1785220582.377693:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97986a00, transno 4295069640 00000004:00080000:1.0:1785220582.377695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae73480, transno 4295069641 00000100:00100000:3.0:1785220582.377697:0:14175:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+13:3653:x1871938510790400:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21072us (21503us total) trans 4295070238 rc 0/0 00000004:00080000:1.0:1785220582.377697:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97984000, transno 4295069642 00000004:00080000:1.0:1785220582.377700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae71f80, transno 4295069643 00000004:00080000:1.0:1785220582.377702:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b0a80, transno 4295069644 00000100:00100000:3.0:1785220582.377705:0:14175:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284822 00000004:00080000:1.0:1785220582.377705:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98add880, transno 4295069645 00000004:00080000:1.0:1785220582.377708:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86310000, transno 4295069646 00000004:00080000:1.0:1785220582.377710:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adf100, transno 4295069647 00000004:00080000:1.0:1785220582.377712:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d699f80, transno 4295069648 00000004:00080000:1.0:1785220582.377715:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b1c00, transno 4295069649 00000004:00080000:1.0:1785220582.377718:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86311c00, transno 4295069650 00000004:00080000:1.0:1785220582.377720:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d975b2300, transno 4295069651 00000004:00080000:1.0:1785220582.377723:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adc380, transno 4295069652 00000004:00080000:1.0:1785220582.377726:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98add180, transno 4295069653 00000004:00080000:1.0:1785220582.377728:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adfb80, transno 4295069654 00000004:00080000:1.0:1785220582.377731:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adea00, transno 4295069655 00000004:00080000:1.0:1785220582.377734:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98addf80, transno 4295069656 00000004:00080000:1.0:1785220582.377737:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94939180, transno 4295069657 00000004:00080000:1.0:1785220582.377739:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493a300, transno 4295069658 00010000:00010000:3.0:1785220582.377740:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000004:00080000:1.0:1785220582.377742:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493a680, transno 4295069659 00000004:00080000:1.0:1785220582.377745:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9a680, transno 4295069660 00000004:00080000:1.0:1785220582.377760:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f0000, transno 4295069661 00000004:00080000:1.0:1785220582.377763:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9b100, transno 4295069662 00000004:00080000:1.0:1785220582.377765:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f1880, transno 4295069663 00000004:00080000:1.0:1785220582.377768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a98700, transno 4295069664 00000004:00080000:1.0:1785220582.377770:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f3100, transno 4295069665 00000004:00080000:1.0:1785220582.377773:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f0700, transno 4295069666 00000004:00080000:1.0:1785220582.377776:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fa300, transno 4295069667 00000004:00080000:1.0:1785220582.377779:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f1500, transno 4295069668 00000004:00080000:1.0:1785220582.377781:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add5880, transno 4295069669 00000004:00080000:1.0:1785220582.377783:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c87b80, transno 4295069670 00000004:00080000:1.0:1785220582.377786:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c87100, transno 4295069671 00000004:00080000:1.0:1785220582.377788:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c86300, transno 4295069672 00000004:00080000:1.0:1785220582.377791:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c87480, transno 4295069673 00000004:00080000:1.0:1785220582.377794:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ac000, transno 4295069674 00000004:00080000:1.0:1785220582.377796:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add6a00, transno 4295069675 00000004:00080000:1.0:1785220582.377799:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add5c00, transno 4295069676 00000004:00080000:1.0:1785220582.377802:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f0e00, transno 4295069677 00000004:00080000:1.0:1785220582.377804:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f1180, transno 4295069678 00000004:00080000:1.0:1785220582.377807:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97986680, transno 4295069679 00000004:00080000:1.0:1785220582.377810:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f1c00, transno 4295069680 00000004:00080000:1.0:1785220582.377812:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f0380, transno 4295069681 00000004:00080000:1.0:1785220582.377815:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f2300, transno 4295069682 00000004:00080000:1.0:1785220582.377818:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f1f80, transno 4295069683 00000004:00080000:1.0:1785220582.377821:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121a300, transno 4295069684 00000004:00080000:1.0:1785220582.377823:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97984a80, transno 4295069685 00000004:00080000:1.0:1785220582.377825:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8adf80, transno 4295069686 00000004:00080000:1.0:1785220582.377828:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8af800, transno 4295069687 00000004:00080000:1.0:1785220582.377830:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8af100, transno 4295069688 00000004:00080000:1.0:1785220582.377832:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ae300, transno 4295069689 00000004:00080000:1.0:1785220582.377834:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a262a00, transno 4295069690 00000004:00080000:1.0:1785220582.377836:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8121a680, transno 4295069691 00000004:00080000:1.0:1785220582.377838:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ad880, transno 4295069692 00000004:00080000:1.0:1785220582.377841:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97843100, transno 4295069693 00000004:00080000:1.0:1785220582.377844:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add7100, transno 4295069694 00000004:00080000:1.0:1785220582.377846:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97842680, transno 4295069695 00000004:00080000:1.0:1785220582.377848:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97840a80, transno 4295069696 00000004:00080000:1.0:1785220582.377849:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add6680, transno 4295069697 00000004:00080000:1.0:1785220582.377852:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96667800, transno 4295069698 00000004:00080000:1.0:1785220582.377854:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add7480, transno 4295069699 00000004:00080000:1.0:1785220582.377856:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a99180, transno 4295069700 00000004:00080000:1.0:1785220582.377858:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d698e00, transno 4295069701 00000004:00080000:1.0:1785220582.377860:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add4700, transno 4295069702 00000004:00080000:1.0:1785220582.377862:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add4000, transno 4295069703 00000004:00080000:1.0:1785220582.377864:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a706a00, transno 4295069704 00000004:00080000:1.0:1785220582.377866:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add4380, transno 4295069705 00000004:00080000:1.0:1785220582.377868:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f8380, transno 4295069706 00000004:00080000:1.0:1785220582.377871:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fb100, transno 4295069707 00000004:00080000:1.0:1785220582.377872:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adf800, transno 4295069708 00000004:00080000:1.0:1785220582.377874:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a705500, transno 4295069709 00000004:00080000:1.0:1785220582.377877:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98add500, transno 4295069710 00000004:00080000:1.0:1785220582.377880:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a704700, transno 4295069711 00000004:00080000:1.0:1785220582.377882:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add5180, transno 4295069712 00000004:00080000:1.0:1785220582.377883:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d699c00, transno 4295069713 00000004:00080000:1.0:1785220582.377886:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4c000, transno 4295069714 00000004:00080000:1.0:1785220582.377888:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c683b80, transno 4295069715 00000004:00080000:1.0:1785220582.377890:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4d500, transno 4295069716 00000004:00080000:1.0:1785220582.377892:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8add4e00, transno 4295069717 00000004:00080000:1.0:1785220582.377894:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4f480, transno 4295069718 00000004:00080000:1.0:1785220582.377897:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9493b100, transno 4295069719 00000004:00080000:1.0:1785220582.377899:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906fbb80, transno 4295069720 00000004:00080000:1.0:1785220582.377902:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906f9c00, transno 4295069721 00000004:00080000:1.0:1785220582.377904:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d311f80, transno 4295069722 00000004:00080000:1.0:1785220582.377907:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998b9f80, transno 4295069723 00000004:00080000:1.0:1785220582.377909:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d313800, transno 4295069724 00000004:00080000:1.0:1785220582.377912:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae70000, transno 4295069725 00000004:00080000:1.0:1785220582.377914:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99271880, transno 4295069726 00000004:00080000:1.0:1785220582.377916:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c85880, transno 4295069727 00000004:00080000:1.0:1785220582.377919:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99272a00, transno 4295069728 00000004:00080000:1.0:1785220582.377921:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d54380, transno 4295069729 00000004:00080000:1.0:1785220582.377923:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c85180, transno 4295069730 00000004:00080000:1.0:1785220582.377925:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4ca80, transno 4295069731 00000004:00080000:1.0:1785220582.377927:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87d46680, transno 4295069732 00000004:00080000:1.0:1785220582.377929:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99273480, transno 4295069733 00000004:00080000:1.0:1785220582.377932:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98aded80, transno 4295069734 00000004:00080000:1.0:1785220582.377934:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b032680, transno 4295069735 00000004:00080000:1.0:1785220582.377937:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b032a00, transno 4295069736 00000004:00080000:1.0:1785220582.377939:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90142a00, transno 4295069737 00000004:00080000:1.0:1785220582.377942:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a704000, transno 4295069738 00000004:00080000:1.0:1785220582.377945:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90142d80, transno 4295069739 00000004:00080000:1.0:1785220582.377947:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987c3b80, transno 4295069740 00000004:00080000:1.0:1785220582.377950:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b033800, transno 4295069741 00000004:00080000:1.0:1785220582.377952:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00b100, transno 4295069742 00000004:00080000:1.0:1785220582.377955:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4ce00, transno 4295069743 00000004:00080000:1.0:1785220582.377958:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b030e00, transno 4295069744 00000004:00080000:1.0:1785220582.377960:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4f800, transno 4295069745 00000004:00080000:1.0:1785220582.377962:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94938700, transno 4295069746 00000004:00080000:1.0:1785220582.377964:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4c700, transno 4295069747 00000004:00080000:1.0:1785220582.377967:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4ea00, transno 4295069748 00000004:00080000:1.0:1785220582.377969:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fad80, transno 4295069749 00000004:00080000:1.0:1785220582.377971:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00ad80, transno 4295069750 00000004:00080000:1.0:1785220582.377973:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94939f80, transno 4295069751 00000004:00080000:1.0:1785220582.377976:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa61f80, transno 4295069752 00000004:00080000:1.0:1785220582.377978:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa62300, transno 4295069753 00000004:00080000:1.0:1785220582.377981:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa60000, transno 4295069754 00000004:00080000:1.0:1785220582.377983:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa61c00, transno 4295069755 00000004:00080000:1.0:1785220582.377985:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa63100, transno 4295069756 00000004:00080000:1.0:1785220582.377988:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa63b80, transno 4295069757 00000004:00080000:1.0:1785220582.377990:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91479500, transno 4295069758 00000004:00080000:1.0:1785220582.377993:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93608700, transno 4295069759 00000004:00080000:1.0:1785220582.377995:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91479f80, transno 4295069760 00000004:00080000:1.0:1785220582.377998:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ac380, transno 4295069761 00000004:00080000:1.0:1785220582.378000:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915af100, transno 4295069762 00000004:00080000:1.0:1785220582.378002:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915aed80, transno 4295069763 00000004:00080000:1.0:1785220582.378005:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91478e00, transno 4295069764 00000004:00080000:1.0:1785220582.378007:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915af480, transno 4295069765 00000004:00080000:1.0:1785220582.378010:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91478380, transno 4295069766 00000004:00080000:1.0:1785220582.378012:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93608a80, transno 4295069767 00000004:00080000:1.0:1785220582.378014:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360a680, transno 4295069768 00000004:00080000:1.0:1785220582.378017:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93608e00, transno 4295069769 00000004:00080000:1.0:1785220582.378019:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360a300, transno 4295069770 00000004:00080000:1.0:1785220582.378022:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00a300, transno 4295069771 00000004:00080000:1.0:1785220582.378024:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360bb80, transno 4295069772 00000004:00080000:1.0:1785220582.378027:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00b480, transno 4295069773 00000004:00080000:1.0:1785220582.378029:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93609880, transno 4295069774 00000004:00080000:1.0:1785220582.378032:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9360ad80, transno 4295069775 00000004:00080000:1.0:1785220582.378034:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b008000, transno 4295069776 00000004:00080000:1.0:1785220582.378036:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b009c00, transno 4295069777 00000004:00080000:1.0:1785220582.378038:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00aa00, transno 4295069778 00000004:00080000:1.0:1785220582.378041:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a99500, transno 4295069779 00000004:00080000:1.0:1785220582.378043:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9a300, transno 4295069780 00000004:00080000:1.0:1785220582.378046:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97843800, transno 4295069781 00000004:00080000:1.0:1785220582.378048:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147aa00, transno 4295069782 00000004:00080000:1.0:1785220582.378051:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa61180, transno 4295069783 00000004:00080000:1.0:1785220582.378053:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa60380, transno 4295069784 00000004:00080000:1.0:1785220582.378056:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa62a00, transno 4295069785 00000004:00080000:1.0:1785220582.378058:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa60700, transno 4295069786 00000004:00080000:1.0:1785220582.378060:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa63480, transno 4295069787 00000004:00080000:1.0:1785220582.378063:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93666a00, transno 4295069788 00000004:00080000:1.0:1785220582.378065:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa62680, transno 4295069789 00000004:00080000:1.0:1785220582.378068:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93608000, transno 4295069790 00000004:00080000:1.0:1785220582.378071:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b009180, transno 4295069791 00000004:00080000:1.0:1785220582.378073:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00b800, transno 4295069792 00000004:00080000:1.0:1785220582.378075:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91478a80, transno 4295069793 00000004:00080000:1.0:1785220582.378077:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b009500, transno 4295069794 00000004:00080000:1.0:1785220582.378080:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbfb80, transno 4295069795 00000004:00080000:1.0:1785220582.378082:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b009880, transno 4295069796 00000004:00080000:1.0:1785220582.378482:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90141f80, transno 4295069797 00000004:00080000:1.0:1785220582.378485:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbc380, transno 4295069798 00000004:00080000:1.0:1785220582.378487:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915aea00, transno 4295069799 00000004:00080000:1.0:1785220582.378490:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90143800, transno 4295069800 00000004:00080000:1.0:1785220582.378492:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915adf80, transno 4295069801 00000004:00080000:1.0:1785220582.378494:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90140000, transno 4295069802 00000004:00080000:1.0:1785220582.378496:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90140a80, transno 4295069803 00000004:00080000:1.0:1785220582.378498:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a5f80, transno 4295069804 00000004:00080000:1.0:1785220582.378500:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8ad180, transno 4295069805 00000004:00080000:1.0:1785220582.378503:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a6a00, transno 4295069806 00000004:00080000:1.0:1785220582.378505:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a6300, transno 4295069807 00000004:00080000:1.0:1785220582.378507:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a5500, transno 4295069808 00000004:00080000:1.0:1785220582.378510:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90142300, transno 4295069809 00000004:00080000:1.0:1785220582.378512:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8aca80, transno 4295069810 00000004:00080000:1.0:1785220582.378514:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a4000, transno 4295069811 00000004:00080000:1.0:1785220582.378516:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a6680, transno 4295069812 00000004:00080000:1.0:1785220582.378519:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a5180, transno 4295069813 00000004:00080000:1.0:1785220582.378521:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a4a80, transno 4295069814 00000004:00080000:1.0:1785220582.378524:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a7480, transno 4295069815 00000004:00080000:1.0:1785220582.378526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97987b80, transno 4295069816 00000004:00080000:1.0:1785220582.378528:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbce00, transno 4295069817 00000004:00080000:1.0:1785220582.378531:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365dc00, transno 4295069818 00000004:00080000:1.0:1785220582.378534:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a7100, transno 4295069819 00000004:00080000:1.0:1785220582.378536:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365f480, transno 4295069820 00000004:00080000:1.0:1785220582.378539:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbc380, transno 4295069821 00000004:00080000:1.0:1785220582.378541:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ed500, transno 4295069822 00000004:00080000:1.0:1785220582.378544:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba5500, transno 4295069823 00000004:00080000:1.0:1785220582.378546:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6edc00, transno 4295069824 00000004:00080000:1.0:1785220582.378549:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6eed80, transno 4295069825 00000004:00080000:1.0:1785220582.378551:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba5880, transno 4295069826 00000004:00080000:1.0:1785220582.378553:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b030380, transno 4295069827 00000004:00080000:1.0:1785220582.378556:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365ca80, transno 4295069828 00000004:00080000:1.0:1785220582.378558:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba4e00, transno 4295069829 00000004:00080000:1.0:1785220582.378561:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365e680, transno 4295069830 00000004:00080000:1.0:1785220582.378564:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365fb80, transno 4295069831 00000004:00080000:1.0:1785220582.378566:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365d880, transno 4295069832 00000004:00080000:1.0:1785220582.378569:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365ea00, transno 4295069833 00000004:00080000:1.0:1785220582.378571:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365d180, transno 4295069834 00000004:00080000:1.0:1785220582.378574:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365f100, transno 4295069835 00000004:00080000:1.0:1785220582.378576:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa1500, transno 4295069836 00000004:00080000:1.0:1785220582.378578:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365df80, transno 4295069837 00000004:00080000:1.0:1785220582.378580:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365c380, transno 4295069838 00000004:00080000:1.0:1785220582.378583:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955ffb80, transno 4295069839 00000004:00080000:1.0:1785220582.378585:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fd500, transno 4295069840 00000004:00080000:1.0:1785220582.378587:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa0700, transno 4295069841 00000004:00080000:1.0:1785220582.378589:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa1180, transno 4295069842 00000004:00080000:1.0:1785220582.378592:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b109500, transno 4295069843 00000004:00080000:1.0:1785220582.378595:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b109f80, transno 4295069844 00000004:00080000:1.0:1785220582.378597:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9365c700, transno 4295069845 00000004:00080000:1.0:1785220582.378599:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b108380, transno 4295069846 00000004:00080000:1.0:1785220582.378601:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbc700, transno 4295069847 00000004:00080000:1.0:1785220582.378604:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93665f80, transno 4295069848 00000004:00080000:1.0:1785220582.378606:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ef480, transno 4295069849 00000004:00080000:1.0:1785220582.378608:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6efb80, transno 4295069850 00000004:00080000:1.0:1785220582.378611:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ee300, transno 4295069851 00000004:00080000:1.0:1785220582.378613:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ed180, transno 4295069852 00000004:00080000:1.0:1785220582.378616:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ec000, transno 4295069853 00000004:00080000:1.0:1785220582.378618:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a7800, transno 4295069854 00000004:00080000:1.0:1785220582.378621:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a5c00, transno 4295069855 00000004:00080000:1.0:1785220582.378623:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6edf80, transno 4295069856 00000004:00080000:1.0:1785220582.378626:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93666d80, transno 4295069857 00000004:00080000:1.0:1785220582.378629:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147ad80, transno 4295069858 00000004:00080000:1.0:1785220582.378632:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91478000, transno 4295069859 00000004:00080000:1.0:1785220582.378634:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147a300, transno 4295069860 00000004:00080000:1.0:1785220582.378637:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93665880, transno 4295069861 00000004:00080000:1.0:1785220582.378639:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93667b80, transno 4295069862 00000004:00080000:1.0:1785220582.378641:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72dc00, transno 4295069863 00000004:00080000:1.0:1785220582.378644:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91479c00, transno 4295069864 00000004:00080000:1.0:1785220582.378646:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f725500, transno 4295069865 00000004:00080000:1.0:1785220582.378649:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f726d80, transno 4295069866 00000004:00080000:1.0:1785220582.378651:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93664000, transno 4295069867 00000004:00080000:1.0:1785220582.378654:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f727800, transno 4295069868 00000004:00080000:1.0:1785220582.378656:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f727100, transno 4295069869 00000004:00080000:1.0:1785220582.378659:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93666680, transno 4295069870 00000004:00080000:1.0:1785220582.378661:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93667100, transno 4295069871 00000004:00080000:1.0:1785220582.378664:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93665180, transno 4295069872 00000004:00080000:1.0:1785220582.378667:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93665c00, transno 4295069873 00000004:00080000:1.0:1785220582.378669:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ad180, transno 4295069874 00000004:00080000:1.0:1785220582.378672:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147b800, transno 4295069875 00000004:00080000:1.0:1785220582.378674:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915adc00, transno 4295069876 00000004:00080000:1.0:1785220582.378677:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be6300, transno 4295069877 00000004:00080000:1.0:1785220582.378679:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b033480, transno 4295069878 00000004:00080000:1.0:1785220582.378682:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c681180, transno 4295069879 00000004:00080000:1.0:1785220582.378684:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be4380, transno 4295069880 00000004:00080000:1.0:1785220582.378687:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90143100, transno 4295069881 00000004:00080000:1.0:1785220582.378689:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb1880, transno 4295069882 00000004:00080000:1.0:1785220582.378692:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90143b80, transno 4295069883 00000004:00080000:1.0:1785220582.378695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90140e00, transno 4295069884 00000004:00080000:1.0:1785220582.378697:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb1500, transno 4295069885 00000004:00080000:1.0:1785220582.378700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb1f80, transno 4295069886 00000004:00080000:1.0:1785220582.378702:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90142680, transno 4295069887 00000004:00080000:1.0:1785220582.378705:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8d880, transno 4295069888 00000004:00080000:1.0:1785220582.378707:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb3480, transno 4295069889 00000004:00080000:1.0:1785220582.378710:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb2a00, transno 4295069890 00000004:00080000:1.0:1785220582.378713:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb0700, transno 4295069891 00000004:00080000:1.0:1785220582.378715:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb3b80, transno 4295069892 00000004:00080000:1.0:1785220582.378718:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e05c00, transno 4295069893 00000004:00080000:1.0:1785220582.378720:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e04e00, transno 4295069894 00000004:00080000:1.0:1785220582.378722:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e05880, transno 4295069895 00000004:00080000:1.0:1785220582.378724:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e06680, transno 4295069896 00000004:00080000:1.0:1785220582.378726:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e07800, transno 4295069897 00000004:00080000:1.0:1785220582.378730:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbd180, transno 4295069898 00000004:00080000:1.0:1785220582.378732:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e05f80, transno 4295069899 00000004:00080000:1.0:1785220582.378735:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99066a00, transno 4295069900 00000004:00080000:1.0:1785220582.378737:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99065500, transno 4295069901 00000004:00080000:1.0:1785220582.378740:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e06300, transno 4295069902 00000004:00080000:1.0:1785220582.378743:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbd500, transno 4295069903 00000004:00080000:1.0:1785220582.378745:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99065c00, transno 4295069904 00000004:00080000:1.0:1785220582.378935:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be5c00, transno 4295069905 00000004:00080000:1.0:1785220582.378937:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ace00, transno 4295069906 00000004:00080000:1.0:1785220582.378939:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be4700, transno 4295069907 00000004:00080000:1.0:1785220582.378942:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99066d80, transno 4295069908 00000004:00080000:1.0:1785220582.378945:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99066680, transno 4295069909 00000004:00080000:1.0:1785220582.378947:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99067800, transno 4295069910 00000004:00080000:1.0:1785220582.378950:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99064380, transno 4295069911 00000004:00080000:1.0:1785220582.378952:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99065180, transno 4295069912 00000004:00080000:1.0:1785220582.378955:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adf480, transno 4295069913 00000004:00080000:1.0:1785220582.378957:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb2300, transno 4295069914 00000004:00080000:1.0:1785220582.378960:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb0000, transno 4295069915 00000004:00080000:1.0:1785220582.378963:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e8a80, transno 4295069916 00000004:00080000:1.0:1785220582.378965:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb3100, transno 4295069917 00000004:00080000:1.0:1785220582.378968:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb2680, transno 4295069918 00000004:00080000:1.0:1785220582.378970:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb1180, transno 4295069919 00000004:00080000:1.0:1785220582.378973:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99064000, transno 4295069920 00000004:00080000:1.0:1785220582.378976:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb3800, transno 4295069921 00000004:00080000:1.0:1785220582.378978:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99064e00, transno 4295069922 00000004:00080000:1.0:1785220582.378981:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5a300, transno 4295069923 00000004:00080000:1.0:1785220582.378984:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ec380, transno 4295069924 00000004:00080000:1.0:1785220582.378986:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa0e00, transno 4295069925 00000004:00080000:1.0:1785220582.378989:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa0000, transno 4295069926 00000004:00080000:1.0:1785220582.378991:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991b480, transno 4295069927 00000004:00080000:1.0:1785220582.378994:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa2a00, transno 4295069928 00000004:00080000:1.0:1785220582.378997:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977f2d80, transno 4295069929 00000004:00080000:1.0:1785220582.378999:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa3800, transno 4295069930 00000004:00080000:1.0:1785220582.379002:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e05180, transno 4295069931 00000004:00080000:1.0:1785220582.379004:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6eea00, transno 4295069932 00000004:00080000:1.0:1785220582.379007:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ae680, transno 4295069933 00000004:00080000:1.0:1785220582.379010:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa2d80, transno 4295069934 00000004:00080000:1.0:1785220582.379013:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa61500, transno 4295069935 00000004:00080000:1.0:1785220582.379015:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa1c00, transno 4295069936 00000004:00080000:1.0:1785220582.379018:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6ece00, transno 4295069937 00000004:00080000:1.0:1785220582.379020:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb22300, transno 4295069938 00000004:00080000:1.0:1785220582.379023:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba6680, transno 4295069939 00000004:00080000:1.0:1785220582.379025:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb23800, transno 4295069940 00000004:00080000:1.0:1785220582.379028:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba7800, transno 4295069941 00000004:00080000:1.0:1785220582.379030:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba7480, transno 4295069942 00000004:00080000:1.0:1785220582.379033:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb23100, transno 4295069943 00000004:00080000:1.0:1785220582.379035:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb64a80, transno 4295069944 00000004:00080000:1.0:1785220582.379037:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa0380, transno 4295069945 00000004:00080000:1.0:1785220582.379039:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb65c00, transno 4295069946 00000004:00080000:1.0:1785220582.379042:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb64700, transno 4295069947 00000004:00080000:1.0:1785220582.379045:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10aa00, transno 4295069948 00000004:00080000:1.0:1785220582.379047:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991aa00, transno 4295069949 00000004:00080000:1.0:1785220582.379049:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991b800, transno 4295069950 00000004:00080000:1.0:1785220582.379052:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b109880, transno 4295069951 00000004:00080000:1.0:1785220582.379054:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10a680, transno 4295069952 00000004:00080000:1.0:1785220582.379056:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b109180, transno 4295069953 00000004:00080000:1.0:1785220582.379059:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b108e00, transno 4295069954 00000004:00080000:1.0:1785220582.379061:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b109c00, transno 4295069955 00000004:00080000:1.0:1785220582.379064:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c681c00, transno 4295069956 00000004:00080000:1.0:1785220582.379066:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba4000, transno 4295069957 00000004:00080000:1.0:1785220582.379069:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be7b80, transno 4295069958 00000004:00080000:1.0:1785220582.379071:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b108a80, transno 4295069959 00000004:00080000:1.0:1785220582.379073:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f727b80, transno 4295069960 00000004:00080000:1.0:1785220582.379075:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa1880, transno 4295069961 00000004:00080000:1.0:1785220582.379077:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e07100, transno 4295069962 00000004:00080000:1.0:1785220582.379079:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa2680, transno 4295069963 00000004:00080000:1.0:1785220582.379081:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72ce00, transno 4295069964 00000004:00080000:1.0:1785220582.379164:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083d180, transno 4295069965 00000004:00080000:1.0:1785220582.379168:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083fb80, transno 4295069966 00000004:00080000:1.0:1785220582.379171:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083c700, transno 4295069967 00000004:00080000:1.0:1785220582.379173:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083ca80, transno 4295069968 00000004:00080000:1.0:1785220582.379176:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97841f80, transno 4295069969 00000004:00080000:1.0:1785220582.379178:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fea00, transno 4295069970 00000004:00080000:1.0:1785220582.379181:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97842d80, transno 4295069971 00000004:00080000:1.0:1785220582.379183:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99919500, transno 4295069972 00000004:00080000:1.0:1785220582.379186:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e06d80, transno 4295069973 00000004:00080000:1.0:1785220582.379188:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb66d80, transno 4295069974 00000004:00080000:1.0:1785220582.379191:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb65180, transno 4295069975 00000004:00080000:1.0:1785220582.379193:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb67480, transno 4295069976 00000004:00080000:1.0:1785220582.379195:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99067480, transno 4295069977 00000004:00080000:1.0:1785220582.379197:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be7800, transno 4295069978 00000004:00080000:1.0:1785220582.379199:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083d500, transno 4295069979 00000004:00080000:1.0:1785220582.379201:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083c000, transno 4295069980 00000004:00080000:1.0:1785220582.379203:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991ad80, transno 4295069981 00000004:00080000:1.0:1785220582.379205:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be6680, transno 4295069982 00000004:00080000:1.0:1785220582.379208:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99919880, transno 4295069983 00000004:00080000:1.0:1785220582.379210:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be6a00, transno 4295069984 00000004:00080000:1.0:1785220582.379212:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7c380, transno 4295069985 00000004:00080000:1.0:1785220582.379215:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94913100, transno 4295069986 00000004:00080000:1.0:1785220582.379217:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94912d80, transno 4295069987 00000100:00100000:0.0:1785220582.379217:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:1.0:1785220582.379220:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94913480, transno 4295069988 00000004:00080000:1.0:1785220582.379222:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94910700, transno 4295069989 00000004:00080000:1.0:1785220582.379224:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94911880, transno 4295069990 00000004:00080000:1.0:1785220582.379226:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94913b80, transno 4295069991 00000004:00080000:1.0:1785220582.379228:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94912a00, transno 4295069992 00000004:00080000:1.0:1785220582.379231:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adf480, transno 4295069993 00000004:00080000:1.0:1785220582.379233:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adfb80, transno 4295069994 00000004:00080000:1.0:1785220582.379235:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adea00, transno 4295069995 00000004:00080000:1.0:1785220582.379237:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adc700, transno 4295069996 00000004:00080000:1.0:1785220582.379239:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97aded80, transno 4295069997 00000004:00080000:1.0:1785220582.379241:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adce00, transno 4295069998 00000004:00080000:1.0:1785220582.379243:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99918700, transno 4295069999 00000004:00080000:1.0:1785220582.379245:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97addc00, transno 4295070000 00000004:00080000:1.0:1785220582.379248:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90141c00, transno 4295070001 00000004:00080000:1.0:1785220582.379250:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e04a80, transno 4295070002 00000100:00100000:0.0:1785220582.379250:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520003968 00000004:00080000:1.0:1785220582.379252:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb36d80, transno 4295070003 00000004:00080000:1.0:1785220582.379254:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb20380, transno 4295070004 00000004:00080000:1.0:1785220582.379257:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb21180, transno 4295070005 00000100:00100000:0.0:1785220582.379258:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520003968 00000004:00080000:1.0:1785220582.379259:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb20a80, transno 4295070006 00000004:00080000:1.0:1785220582.379260:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94910a80, transno 4295070007 00000004:00080000:1.0:1785220582.379262:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa3100, transno 4295070008 00000004:00080000:1.0:1785220582.379265:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa3480, transno 4295070009 00000004:00080000:1.0:1785220582.379267:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb65500, transno 4295070010 00000004:00080000:1.0:1785220582.379269:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa1f80, transno 4295070011 00000004:00080000:1.0:1785220582.379271:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b49c00, transno 4295070012 00000004:00080000:1.0:1785220582.379274:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb21f80, transno 4295070013 00000004:00080000:1.0:1785220582.379276:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94912680, transno 4295070014 00000004:00080000:1.0:1785220582.379278:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083e300, transno 4295070015 00000004:00080000:1.0:1785220582.379280:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083f100, transno 4295070016 00000004:00080000:1.0:1785220582.379282:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9083dc00, transno 4295070017 00000004:00080000:1.0:1785220582.379284:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99064a80, transno 4295070018 00000004:00080000:1.0:1785220582.379287:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbd500, transno 4295070019 00000004:00080000:1.0:1785220582.379289:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99067b80, transno 4295070020 00010000:00010000:0.0:1785220582.379289:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005b3cf899/0xf62499ab92793696 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb7b expref: 40 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379291:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbf100, transno 4295070021 00000004:00080000:1.0:1785220582.379293:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601c000, transno 4295070022 00000004:00080000:1.0:1785220582.379296:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbc000, transno 4295070023 00000004:00080000:1.0:1785220582.379298:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10a300, transno 4295070024 00000004:00080000:1.0:1785220582.379301:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb1c00, transno 4295070025 00000004:00080000:1.0:1785220582.379304:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94911180, transno 4295070026 00000004:00080000:1.0:1785220582.379306:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99066300, transno 4295070027 00000004:00080000:1.0:1785220582.379308:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4b800, transno 4295070028 00000004:00080000:1.0:1785220582.379310:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4a300, transno 4295070029 00000004:00080000:1.0:1785220582.379312:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93666300, transno 4295070030 00000004:00080000:1.0:1785220582.379315:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601c700, transno 4295070031 00000004:00080000:1.0:1785220582.379317:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4ad80, transno 4295070032 00000004:00080000:1.0:1785220582.379319:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b48700, transno 4295070033 00000004:00080000:1.0:1785220582.379322:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b48a80, transno 4295070034 00000004:00080000:1.0:1785220582.379324:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955ff480, transno 4295070035 00000004:00080000:1.0:1785220582.379326:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94acaa00, transno 4295070036 00000100:00100000:0.0:1785220582.379328:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100760 00000004:00080000:1.0:1785220582.379329:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac9880, transno 4295070037 00000004:00080000:1.0:1785220582.379331:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98405880, transno 4295070038 00000004:00080000:1.0:1785220582.379334:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98406300, transno 4295070039 00000100:00100000:0.0:1785220582.379336:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+40:2406:x1871938520003968:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000004:00080000:1.0:1785220582.379337:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98407100, transno 4295070040 00000004:00080000:1.0:1785220582.379339:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac9500, transno 4295070041 00000004:00080000:1.0:1785220582.379342:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98404380, transno 4295070042 00000004:00080000:1.0:1785220582.379344:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac8700, transno 4295070043 00000004:00080000:1.0:1785220582.379346:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98405500, transno 4295070044 00000004:00080000:1.0:1785220582.379348:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98405c00, transno 4295070045 00010000:00010000:0.0:1785220582.379349:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00080000:1.0:1785220582.379350:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98404a80, transno 4295070046 00000004:00080000:1.0:1785220582.379352:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98404700, transno 4295070047 00000004:00080000:1.0:1785220582.379354:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98404e00, transno 4295070048 00010000:00010000:0.0:1785220582.379354:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005b3cf899/0xf62499ab92793696 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb7b expref: 40 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379357:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98407800, transno 4295070049 00000004:00080000:1.0:1785220582.379359:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0f100, transno 4295070050 00000004:00080000:1.0:1785220582.379361:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98407b80, transno 4295070051 00000004:00080000:1.0:1785220582.379363:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0d880, transno 4295070052 00000004:00080000:1.0:1785220582.379366:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0f480, transno 4295070053 00010000:00010000:0.0:1785220582.379366:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005b3cf899/0xf62499ab92793696 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb7b expref: 40 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379368:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0fb80, transno 4295070054 00000004:00080000:1.0:1785220582.379371:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0d180, transno 4295070055 00000004:00080000:1.0:1785220582.379373:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a84000, transno 4295070056 00000004:00080000:1.0:1785220582.379375:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c3800, transno 4295070057 00000004:00080000:1.0:1785220582.379377:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c2d80, transno 4295070058 00000004:00080000:1.0:1785220582.379379:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c1180, transno 4295070059 00000004:00080000:1.0:1785220582.379381:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10bb80, transno 4295070060 00000004:00080000:1.0:1785220582.379384:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b1ec700, transno 4295070061 00000004:00080000:1.0:1785220582.379386:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c1500, transno 4295070062 00010000:00010000:0.0:1785220582.379386:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b3cf899/0xf62499ab92793696 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb7b expref: 40 pid: 14247 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379389:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c2680, transno 4295070063 00000004:00080000:1.0:1785220582.379391:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c1c00, transno 4295070064 00000004:00080000:1.0:1785220582.379393:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c3100, transno 4295070065 00000004:00080000:1.0:1785220582.379396:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a85c00, transno 4295070066 00000004:00080000:1.0:1785220582.379398:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601f480, transno 4295070067 00000004:00080000:1.0:1785220582.379400:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a84700, transno 4295070068 00000004:00080000:1.0:1785220582.379402:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a86d80, transno 4295070069 00010000:00010000:0.0:1785220582.379403:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7342:0x0].0x0 since=0s v=(1 0) 00000004:00080000:1.0:1785220582.379405:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4b480, transno 4295070070 00000004:00080000:1.0:1785220582.379408:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a86300, transno 4295070071 00010000:00010000:0.0:1785220582.379408:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7342:0x0].0x0 since=0s v=(1 0) 00000004:00080000:1.0:1785220582.379411:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601ea00, transno 4295070072 00000004:00080000:1.0:1785220582.379413:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be5500, transno 4295070073 00010000:00010000:0.0:1785220582.379414:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379416:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c3480, transno 4295070074 00000004:00080000:1.0:1785220582.379418:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be5f80, transno 4295070075 00000004:00080000:1.0:1785220582.379421:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7f800, transno 4295070076 00000004:00080000:1.0:1785220582.379423:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7ca80, transno 4295070077 00010000:00010000:0.0:1785220582.379423:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379426:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7fb80, transno 4295070078 00000004:00080000:1.0:1785220582.379428:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7df80, transno 4295070079 00000004:00080000:1.0:1785220582.379430:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac9c00, transno 4295070080 00000004:00080000:1.0:1785220582.379432:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7dc00, transno 4295070081 00000004:00080000:1.0:1785220582.379434:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7f100, transno 4295070082 00000004:00080000:1.0:1785220582.379436:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac9180, transno 4295070083 00000004:00080000:1.0:1785220582.379438:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbfb80, transno 4295070084 00000004:00080000:1.0:1785220582.379440:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac8a80, transno 4295070085 00000004:00080000:1.0:1785220582.379442:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94acb480, transno 4295070086 00000004:00080000:1.0:1785220582.379444:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7d500, transno 4295070087 00000004:00080000:1.0:1785220582.379447:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbe680, transno 4295070088 00000004:00080000:1.0:1785220582.379449:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbca80, transno 4295070089 00000004:00080000:1.0:1785220582.379451:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbe300, transno 4295070090 00010000:00010000:0.0:1785220582.379451:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1785220582.379454:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c0a80, transno 4295070091 00000004:00080000:1.0:1785220582.379456:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c1880, transno 4295070092 00000004:00080000:1.0:1785220582.379458:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f725880, transno 4295070093 00000004:00080000:1.0:1785220582.379460:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94acb100, transno 4295070094 00000004:00080000:1.0:1785220582.379463:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a84a80, transno 4295070095 00000004:00080000:1.0:1785220582.379465:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b1ee300, transno 4295070096 00000004:00080000:1.0:1785220582.379467:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a86680, transno 4295070097 00010000:00010000:0.0:1785220582.379467:0:14176:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379470:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601ca80, transno 4295070098 00000004:00080000:1.0:1785220582.379472:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601df80, transno 4295070099 00000004:00080000:1.0:1785220582.379475:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be4a80, transno 4295070100 00000004:00080000:1.0:1785220582.379477:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601fb80, transno 4295070101 00000004:00080000:1.0:1785220582.379480:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99919180, transno 4295070102 00010000:00010000:0.0:1785220582.379480:0:14176:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379482:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99918000, transno 4295070103 00000004:00080000:1.0:1785220582.379485:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99918a80, transno 4295070104 00000004:00080000:1.0:1785220582.379487:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e8601dc00, transno 4295070105 00000004:00080000:1.0:1785220582.379489:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e07480, transno 4295070106 00010000:00010000:0.0:1785220582.379489:0:14176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379491:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d967c3b80, transno 4295070107 00000004:00080000:1.0:1785220582.379493:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7c700, transno 4295070108 00000004:00080000:1.0:1785220582.379496:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9991bb80, transno 4295070109 00010000:00010000:0.0:1785220582.379497:0:14176:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220582.379499:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7d880, transno 4295070110 00000004:00080000:1.0:1785220582.379501:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99919c00, transno 4295070111 00000004:00080000:1.0:1785220582.379504:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99be4e00, transno 4295070112 00000004:00080000:1.0:1785220582.379506:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7e680, transno 4295070113 00000004:00080000:1.0:1785220582.379508:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7ea00, transno 4295070114 00000004:00080000:1.0:1785220582.379510:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94acb800, transno 4295070115 00000004:00080000:1.0:1785220582.379513:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7d180, transno 4295070116 00010000:00010000:0.0:1785220582.379513:0:14176:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1785220582.379515:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7ed80, transno 4295070117 00000004:00080000:1.0:1785220582.379517:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94acbb80, transno 4295070118 00000004:00080000:1.0:1785220582.379519:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93d7c000, transno 4295070119 00000004:00080000:1.0:1785220582.379521:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b49180, transno 4295070120 00000004:00080000:1.0:1785220582.379524:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ac9f80, transno 4295070121 00000004:00080000:1.0:1785220582.379526:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb20700, transno 4295070122 00000004:00080000:1.0:1785220582.379528:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aca680, transno 4295070123 00000004:00080000:1.0:1785220582.379530:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fc000, transno 4295070124 00000004:00080000:1.0:1785220582.379533:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb23480, transno 4295070125 00000004:00080000:1.0:1785220582.379535:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fe680, transno 4295070126 00000004:00080000:1.0:1785220582.379537:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fdf80, transno 4295070127 00000004:00080000:1.0:1785220582.379540:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0dc00, transno 4295070128 00000004:00080000:1.0:1785220582.379542:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99065880, transno 4295070129 00000004:00080000:1.0:1785220582.379545:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b1edf80, transno 4295070130 00000004:00080000:1.0:1785220582.379547:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fc700, transno 4295070131 00000004:00080000:1.0:1785220582.379550:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91581f80, transno 4295070132 00000004:00080000:1.0:1785220582.379552:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91583480, transno 4295070133 00000004:00080000:1.0:1785220582.379554:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fca80, transno 4295070134 00000004:00080000:1.0:1785220582.379556:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a87800, transno 4295070135 00000004:00080000:1.0:1785220582.379558:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ac700, transno 4295070136 00000004:00080000:1.0:1785220582.379561:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fd880, transno 4295070137 00000004:00080000:1.0:1785220582.379563:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ad880, transno 4295070138 00000004:00080000:1.0:1785220582.379565:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ae300, transno 4295070139 00000004:00080000:1.0:1785220582.379567:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ad500, transno 4295070140 00000004:00080000:1.0:1785220582.379570:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915ac000, transno 4295070141 00000004:00080000:1.0:1785220582.379572:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915afb80, transno 4295070142 00000004:00080000:1.0:1785220582.379574:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b4aa00, transno 4295070143 00000004:00080000:1.0:1785220582.379576:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97840380, transno 4295070144 00000004:00080000:1.0:1785220582.379578:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c681500, transno 4295070145 00000004:00080000:1.0:1785220582.379581:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97841500, transno 4295070146 00000004:00080000:1.0:1785220582.379583:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c683800, transno 4295070147 00000004:00080000:1.0:1785220582.379585:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f727480, transno 4295070148 00000004:00080000:1.0:1785220582.379588:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c682d80, transno 4295070149 00000004:00080000:1.0:1785220582.379590:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b48e00, transno 4295070150 00000004:00080000:1.0:1785220582.379593:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b49880, transno 4295070151 00000004:00080000:1.0:1785220582.379595:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99065f80, transno 4295070152 00000004:00080000:1.0:1785220582.379597:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72d180, transno 4295070153 00000004:00080000:1.0:1785220582.379600:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72c380, transno 4295070154 00000004:00080000:1.0:1785220582.379602:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72f100, transno 4295070155 00000004:00080000:1.0:1785220582.379605:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72ea00, transno 4295070156 00000004:00080000:1.0:1785220582.379607:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72e680, transno 4295070157 00000004:00080000:1.0:1785220582.379609:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915af800, transno 4295070158 00000004:00080000:1.0:1785220582.379612:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d72f800, transno 4295070159 00000004:00080000:1.0:1785220582.379614:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10b480, transno 4295070160 00000004:00080000:1.0:1785220582.379617:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b10b800, transno 4295070161 00000004:00080000:1.0:1785220582.379620:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89efe300, transno 4295070162 00000004:00080000:1.0:1785220582.379622:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b108000, transno 4295070163 00000004:00080000:1.0:1785220582.379625:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89efea00, transno 4295070164 00000004:00080000:1.0:1785220582.379627:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89efca80, transno 4295070165 00000004:00080000:1.0:1785220582.379629:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89fb0e00, transno 4295070166 00000004:00080000:1.0:1785220582.379631:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adca80, transno 4295070167 00000004:00080000:1.0:1785220582.379633:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adc000, transno 4295070168 00000004:00080000:1.0:1785220582.379635:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97addf80, transno 4295070169 00000004:00080000:1.0:1785220582.379638:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97add500, transno 4295070170 00000004:00080000:1.0:1785220582.379640:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbe680, transno 4295070171 00000004:00080000:1.0:1785220582.379643:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0df80, transno 4295070172 00000004:00080000:1.0:1785220582.379645:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d312a00, transno 4295070173 00000004:00080000:1.0:1785220582.379648:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955ff800, transno 4295070174 00000004:00080000:1.0:1785220582.379650:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fe300, transno 4295070175 00000004:00080000:1.0:1785220582.379653:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998b8e00, transno 4295070176 00000004:00080000:1.0:1785220582.379655:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbf800, transno 4295070177 00000004:00080000:1.0:1785220582.379658:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96667b80, transno 4295070178 00000004:00080000:1.0:1785220582.379660:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d955fc380, transno 4295070179 00000004:00080000:1.0:1785220582.379663:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97841c00, transno 4295070180 00000004:00080000:1.0:1785220582.379666:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93762680, transno 4295070181 00000004:00080000:1.0:1785220582.379668:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0c700, transno 4295070182 00000004:00080000:1.0:1785220582.379671:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f9180, transno 4295070183 00000004:00080000:1.0:1785220582.379673:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921f8000, transno 4295070184 00000004:00080000:1.0:1785220582.379675:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fbb80, transno 4295070185 00000004:00080000:1.0:1785220582.379678:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fa680, transno 4295070186 00000004:00080000:1.0:1785220582.379680:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fb480, transno 4295070187 00000004:00080000:1.0:1785220582.379683:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0e680, transno 4295070188 00000004:00080000:1.0:1785220582.379685:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d921fb800, transno 4295070189 00000004:00080000:1.0:1785220582.379688:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95a0ce00, transno 4295070190 00000004:00080000:1.0:1785220582.379691:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d312300, transno 4295070191 00000004:00080000:1.0:1785220582.379693:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dbd880, transno 4295070192 00000004:00080000:1.0:1785220582.379695:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f724000, transno 4295070193 00000004:00080000:1.0:1785220582.379697:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90cbd880, transno 4295070194 00000004:00080000:1.0:1785220582.379700:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89efd180, transno 4295070195 00000004:00080000:1.0:1785220582.379703:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adc700, transno 4295070196 00000004:00080000:1.0:1785220582.379705:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93760e00, transno 4295070197 00000004:00080000:1.0:1785220582.379708:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4c380, transno 4295070198 00000004:00080000:1.0:1785220582.379710:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761500, transno 4295070199 00000004:00080000:1.0:1785220582.379713:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93762a00, transno 4295070200 00000004:00080000:1.0:1785220582.379714:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761180, transno 4295070201 00000004:00080000:1.0:1785220582.379717:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d998bb480, transno 4295070202 00000004:00080000:1.0:1785220582.379719:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93761f80, transno 4295070203 00000004:00080000:1.0:1785220582.379721:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d915aca80, transno 4295070204 00000004:00080000:1.0:1785220582.379723:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00a680, transno 4295070205 00000004:00080000:1.0:1785220582.379726:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9147b100, transno 4295070206 00000004:00080000:1.0:1785220582.379728:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d311880, transno 4295070207 00000004:00080000:1.0:1785220582.379731:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a6d80, transno 4295070208 00000004:00080000:1.0:1785220582.379733:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d923a7b80, transno 4295070209 00000004:00080000:1.0:1785220582.379735:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a98380, transno 4295070210 00000004:00080000:1.0:1785220582.379738:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a9bb80, transno 4295070211 00000004:00080000:1.0:1785220582.379740:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba7100, transno 4295070212 00000004:00080000:1.0:1785220582.379743:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97a98e00, transno 4295070213 00000004:00080000:1.0:1785220582.379745:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa2300, transno 4295070214 00000004:00080000:1.0:1785220582.379758:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba6d80, transno 4295070215 00000004:00080000:1.0:1785220582.379761:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97985180, transno 4295070216 00000004:00080000:1.0:1785220582.379763:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97986300, transno 4295070217 00000004:00080000:1.0:1785220582.379765:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94aa3b80, transno 4295070218 00000004:00080000:1.0:1785220582.379768:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfc4ed80, transno 4295070219 00000004:00080000:1.0:1785220582.379771:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89eff800, transno 4295070220 00000004:00080000:1.0:1785220582.379773:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90141180, transno 4295070221 00000004:00080000:1.0:1785220582.379775:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90140380, transno 4295070222 00000004:00080000:1.0:1785220582.379777:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90141500, transno 4295070223 00000004:00080000:1.0:1785220582.379780:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89efc000, transno 4295070224 00000004:00080000:1.0:1785220582.379782:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8c700, transno 4295070225 00000004:00080000:1.0:1785220582.379784:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8ea00, transno 4295070226 00000004:00080000:1.0:1785220582.379787:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8ed80, transno 4295070227 00000004:00080000:1.0:1785220582.379789:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90ba7b80, transno 4295070228 00000004:00080000:1.0:1785220582.379791:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8dc00, transno 4295070229 00000004:00080000:1.0:1785220582.379793:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8adc00, transno 4295070230 00000004:00080000:1.0:1785220582.379796:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d8aed80, transno 4295070231 00000004:00080000:1.0:1785220582.379798:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e8f800, transno 4295070232 00000004:00080000:1.0:1785220582.379801:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad2d80, transno 4295070233 00000004:00080000:1.0:1785220582.379803:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad2680, transno 4295070234 00000004:00080000:1.0:1785220582.379805:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad2300, transno 4295070235 00000004:00080000:1.0:1785220582.379807:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985ebb80, transno 4295070236 00000100:00100000:1.0:1785220582.379816:0:3655:0:(client.c:3156:ptlrpc_free_committed()) @@@ stopping search req@ffff960d98c84000 x1871938510790528/t4295070237(4295070237) o6->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1785220598 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1785220582.379830:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c84000/2, rc 0, transno 4295070237 00000100:00100000:1.0:1785220582.379835:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510790528:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.379926:0:14176:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_037:lustre-MDT0000-mdtlov_UUID+191:3655:x1871938510791040:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22035us (22901us total) trans 4295070813 rc 0/0 00000100:00100000:0.0:1785220582.379934:0:14176:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284824 00000100:00100000:0.0:1785220582.379976:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+39:2406:x1871938520003968:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 641us (763us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.379982:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100760 00010000:00010000:2.0:1785220582.380216:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380227:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380241:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380248:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380268:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.380355:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d923a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+191:3652:x1871938510791168:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22154us (23095us total) trans 4295070814 rc 0/0 00000100:00100000:2.0:1785220582.380362:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284825 00010000:00010000:2.0:1785220582.380395:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.380402:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380410:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.380414:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380423:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380437:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380444:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095f9a9bf/0xf62499ab927a5b0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380461:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.380470:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380476:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.380479:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380487:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.380495:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.380496:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380503:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eefecec7/0xf62499ab927a5b15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380515:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.380517:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.380518:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.380553:0:14149:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003be201d2/0xf62499ab927936ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbb3 expref: 38 pid: 14172 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.380566:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0000_UUID:14149:1871938510792064:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.380581:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380591:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380598:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380607:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380614:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.380628:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.381258:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381269:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.381273:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381280:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381293:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381300:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5b00 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381318:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.381321:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381326:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.381329:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381336:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381344:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381351:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8347405/0xf62499ab927a5b23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.381361:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.381363:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.381364:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.381427:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.381459:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a261880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+191:3655:x1871938510791808:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21220us (21308us total) trans 4295070815 rc 0/0 00000100:00100000:2.0:1785220582.381467:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284827 00000100:00100000:0.0:1785220582.381497:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94d54a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0001_UUID:14158:1871938510792576:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.381498:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510792192:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.381510:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.381510:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.381529:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004224 00000100:00100000:2.0:1785220582.381534:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004224 00010000:00010000:2.0:1785220582.381556:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007fad1ac5/0xf62499ab927936c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fba5 expref: 38 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.381590:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100761 00000100:00100000:2.0:1785220582.381599:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+38:2406:x1871938520004224:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.381611:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.381617:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007fad1ac5/0xf62499ab927936c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fba5 expref: 38 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381627:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007fad1ac5/0xf62499ab927936c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fba5 expref: 38 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381643:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007fad1ac5/0xf62499ab927936c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fba5 expref: 38 pid: 14197 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381655:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7345:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.381660:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7345:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.381665:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381674:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381694:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.381721:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+37:2406:x1871938520004224:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 124us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.381729:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100761 00010000:00010000:2.0:1785220582.381761:0:14252:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381770:0:14252:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381779:0:14252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381787:0:14252:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381805:0:14252:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.381866:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.381870:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.381878:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.381881:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.381889:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.381900:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.381907:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5af9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.381923:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.381925:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.381927:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.381928:0:14252:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_078:lustre-MDT0000-mdtlov_UUID+13:3652:x1871938510790272:12345-0@lo:6:osp-syn-1-0.0 Request processed in 25469us (25894us total) trans 4295070239 rc 0/0 00000100:00100000:2.0:1785220582.381935:0:14252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284821 00000100:00100000:0.0:1785220582.381948:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.381957:0:14198:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381966:0:14198:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381975:0:14198:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.381982:0:14198:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.381995:0:14261:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_084:lustre-OST0001_UUID:14261:1871938510793216:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.381999:0:14198:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.382009:0:14261:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.382025:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004608 00000100:00100000:0.0:1785220582.382029:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004608 00010000:00010000:0.0:1785220582.382044:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000077cbc2ad/0xf62499ab927934b3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f998 expref: 68 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.382068:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100762 00000100:00100000:2.0:1785220582.382075:0:14198:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ebfad0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_055:lustre-MDT0000-mdtlov_UUID+191:3655:x1871938510792704:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20149us (20367us total) trans 4295070816 rc 0/0 00000100:00100000:0.0:1785220582.382075:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+68:2407:x1871938520004608:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220582.382081:0:14198:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284830 00010000:00010000:0.0:1785220582.382103:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.382108:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000077cbc2ad/0xf62499ab927934b3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f998 expref: 68 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.382115:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.382119:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.382126:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.382129:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.382137:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.382147:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.382155:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023df6f47/0xf62499ab927a5b38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.382169:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.382171:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.382173:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.382179:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000077cbc2ad/0xf62499ab927934b3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f998 expref: 68 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382194:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077cbc2ad/0xf62499ab927934b3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f998 expref: 68 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382203:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.382206:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.382209:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382216:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382232:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.382242:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.382245:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:0.0:1785220582.382254:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.382259:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510794112:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.382261:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382267:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382279:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.382292:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.382355:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d94d55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+13:3653:x1871938510792448:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20879us (20933us total) trans 4295070240 rc 0/0 00000100:00100000:0.0:1785220582.382362:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284829 00000100:00100000:0.0:1785220582.382373:0:14158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510794112 00000100:00100000:0.0:1785220582.382376:0:14158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510794112 00000100:00100000:0.0:1785220582.382397:0:14158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284836 00000100:00100000:0.0:1785220582.382402:0:14158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+191:3654:x1871938510794112:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.382422:0:14158:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29513 count 1 00010000:00010000:0.0:1785220582.382440:0:14158:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382453:0:14158:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009ffdc4a3/0xf62499ab927936f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbdd expref: 36 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382459:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7349:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.382462:0:14158:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7349:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.382466:0:14158:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382484:0:14158:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009ffdc4a3/0xf62499ab927936f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbdd expref: 36 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.382494:0:14158:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009ffdc4a3/0xf62499ab927936f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbdd expref: 36 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.382504:0:14158:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510794240:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.382542:0:14158:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.382568:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+67:2407:x1871938520004608:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 494us (620us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.382575:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100762 00010000:00010000:3.0:1785220582.382771:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.382778:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.382785:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.382788:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.382795:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.382806:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.382813:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e343f9d2/0xf62499ab927a5b31 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.382829:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.382830:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.382832:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.382855:0:14221:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_067:lustre-OST0001_UUID:14221:1871938510793472:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.382867:0:14221:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.383258:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.383312:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d923a4380/2, rc 0, transno 4295070814 00000100:00100000:3.0:1785220582.383324:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d923a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510791168:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.383364:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c85f80/2, rc 0, transno 4295070239 00000100:00100000:3.0:1785220582.383368:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c85f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510790272:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.383378:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510794368:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.383406:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.383429:0:14251:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510794368 00000100:00100000:3.0:1785220582.383432:0:14251:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510794368 00000100:00100000:3.0:1785220582.383457:0:14251:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284837 00000100:00100000:3.0:1785220582.383463:0:14251:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+192:3652:x1871938510794368:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.383477:0:14251:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29514 count 1 00010000:00010000:3.0:1785220582.383492:0:14251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.383504:0:14251:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006bff9206/0xf62499ab92793706 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbeb expref: 36 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.383512:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x734a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.383515:0:14251:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.383518:0:14251:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.383535:0:14251:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006bff9206/0xf62499ab92793706 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbeb expref: 36 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.383544:0:14251:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006bff9206/0xf62499ab92793706 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbeb expref: 36 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.383554:0:14251:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510794496:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.383587:0:14251:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.383786:0:14183:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_041:lustre-OST0001_UUID:14183:1871938510793728:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.383801:0:14183:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.383864:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0000_UUID:14188:1871938510793984:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.383876:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.383927:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.383943:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004736 00000100:00100000:0.0:1785220582.383947:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004736 00010000:00010000:0.0:1785220582.383965:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000498038a1/0xf62499ab927936b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb97 expref: 37 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.383990:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100763 00000100:00100000:0.0:1785220582.383996:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+37:2406:x1871938520004736:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.384004:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.384009:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000498038a1/0xf62499ab927936b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb97 expref: 37 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384018:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000498038a1/0xf62499ab927936b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb97 expref: 37 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384033:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000498038a1/0xf62499ab927936b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb97 expref: 37 pid: 14210 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384044:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7344:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.384047:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7344:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.384053:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384061:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384081:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:3.0:1785220582.384224:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94d55c00/2, rc 0, transno 4295070816 00000100:00100000:3.0:1785220582.384233:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94d55c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510792704:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.384268:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1785220582.384276:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510794624:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.384304:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220582.384342:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a261c00/2, rc 0, transno 4295070815 00000100:00100000:3.0:1785220582.384349:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a261c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510791808:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.384396:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d95a0c000/2, rc 0, transno 4295070813 00000100:00100000:2.0:1785220582.384401:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510794752:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.384403:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d95a0c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510791040:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.384427:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510794880:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.384428:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.384450:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.384916:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2406:x1871938520004736:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 917us (986us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.384924:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100763 00010000:00010000:0.0:1785220582.384949:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384959:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384968:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.384975:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.385000:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.385843:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a260700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+193:3652:x1871938510791552:12345-0@lo:6:osp-syn-0-0.0 Request processed in 25401us (25462us total) trans 4295070817 rc 0/0 00000100:00100000:0.0:1785220582.385852:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284828 00010000:00010000:2.0:1785220582.385857:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.385863:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.385871:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:0.0:1785220582.385871:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510794624 00010000:00010000:2.0:1785220582.385876:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.385877:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510794624 00010000:00010000:2.0:1785220582.385885:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.385901:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.385905:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510794752 00000100:00100000:0.0:1785220582.385907:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510794752 00010000:00010000:2.0:1785220582.385910:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5b2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.385920:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510794880 00000100:00100000:0.0:1785220582.385922:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510794880 00010000:00010000:2.0:1785220582.385932:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.385934:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.385935:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.385940:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284838 00000100:00100000:0.0:1785220582.385946:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+195:3652:x1871938510794624:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.385968:0:8442:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29515 count 1 00010000:00010000:0.0:1785220582.385986:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386000:0:8442:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f02ec828/0xf62499ab92793714 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf9 expref: 35 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386007:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x734b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.386011:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.386015:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386179:0:8442:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f02ec828/0xf62499ab92793714 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf9 expref: 35 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386191:0:8442:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f02ec828/0xf62499ab92793714 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf9 expref: 35 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.386203:0:8442:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510795008:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.386247:0:8442:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.386263:0:14204:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284839 00000100:00100000:0.0:1785220582.386270:0:14204:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+195:3654:x1871938510794752:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.386280:0:14204:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29516 count 1 00010000:00010000:0.0:1785220582.386291:0:14204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386301:0:14204:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f21416d4/0xf62499ab92793722 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc07 expref: 35 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386312:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x734c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.386314:0:14204:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.386317:0:14204:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386332:0:14204:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f21416d4/0xf62499ab92793722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc07 expref: 35 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386341:0:14204:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f21416d4/0xf62499ab92793722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc07 expref: 35 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.386349:0:14204:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a704e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510795136:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.386368:0:14204:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.386383:0:14256:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284840 00000100:00100000:0.0:1785220582.386389:0:14256:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+195:3655:x1871938510794880:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.386399:0:14256:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29517 count 1 00010000:00010000:0.0:1785220582.386409:0:14256:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386420:0:14256:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000be509311/0xf62499ab92793730 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc15 expref: 35 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386426:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x734d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.386429:0:14256:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.386433:0:14256:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386448:0:14256:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000be509311/0xf62499ab92793730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc15 expref: 35 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.386456:0:14256:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000be509311/0xf62499ab92793730 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc15 expref: 35 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.386465:0:14256:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510795264:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.386485:0:14256:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.386527:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.386573:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.386763:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.386795:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.386899:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510793088:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.386913:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.386950:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000004:00080000:0.0:1785220582.386981:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94d54700/2, rc 0, transno 4295070240 00000100:00100000:0.0:1785220582.386988:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94d54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510792448:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.386996:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d98c84700/2, rc 0, transno 4295070238 00000100:00100000:0.0:1785220582.386999:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d98c84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510790400:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.387269:0:14166:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.387278:0:14166:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.387288:0:14166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.387295:0:14166:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.387308:0:14166:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.387441:0:14166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b0e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_028:lustre-MDT0000-mdtlov_UUID+196:3655:x1871938510791680:12345-0@lo:6:osp-syn-0-0.0 Request processed in 27239us (27421us total) trans 4295070818 rc 0/0 00000100:00100000:0.0:1785220582.387448:0:14166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284826 00000004:00080000:3.0:1785220582.387937:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a262680/2, rc 0, transno 4295070817 00000100:00100000:3.0:1785220582.387949:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a262680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510791552:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.387967:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520005120 00000100:00100000:3.0:1785220582.387973:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520005120 00010000:00010000:3.0:1785220582.388001:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f86764e3/0xf62499ab927934dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c2 expref: 67 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.388005:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004864 00000100:00100000:2.0:1785220582.388009:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004864 00010000:00010000:2.0:1785220582.388026:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000169dba9e/0xf62499ab927934a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f98a expref: 68 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.388033:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520005248 00000100:00100000:3.0:1785220582.388035:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520005248 00010000:00010000:3.0:1785220582.388048:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000014d44ab9/0xf62499ab927936ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbcf expref: 36 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.388048:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520004992 00000100:00100000:2.0:1785220582.388050:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520004992 00010000:00010000:2.0:1785220582.388060:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ade06539/0xf62499ab927934cf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9b4 expref: 69 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.388063:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520005376 00000100:00100000:3.0:1785220582.388066:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520005376 00010000:00010000:3.0:1785220582.388076:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000094d2bc9e/0xf62499ab927934c1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9a6 expref: 70 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.388078:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100764 00000100:00100000:2.0:1785220582.388102:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+70:2406:x1871938520005120:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.388110:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100765 00010000:00010000:2.0:1785220582.388114:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.388119:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f86764e3/0xf62499ab927934dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c2 expref: 70 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.388128:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+70:2406:x1871938520004864:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.388130:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f86764e3/0xf62499ab927934dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c2 expref: 70 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388135:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.388138:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000169dba9e/0xf62499ab927934a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f98a expref: 70 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388147:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000169dba9e/0xf62499ab927934a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f98a expref: 70 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.388151:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f86764e3/0xf62499ab927934dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9c2 expref: 70 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388158:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000169dba9e/0xf62499ab927934a5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f98a expref: 70 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388167:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.388167:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7322:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.388171:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x731e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.388171:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7322:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.388178:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.388178:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388188:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.388188:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388207:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.388207:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.388269:0:14183:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.388279:0:14183:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.388286:0:14183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.388293:0:14183:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.388312:0:14183:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.388438:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+68:2406:x1871938520004864:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 321us (1863us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.388445:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100765 00000100:00100000:3.0:1785220582.388458:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100766 00000100:00100000:3.0:1785220582.388464:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2404:x1871938520005248:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.388471:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.388475:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000014d44ab9/0xf62499ab927936ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbcf expref: 36 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388484:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000014d44ab9/0xf62499ab927936ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbcf expref: 36 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388496:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000014d44ab9/0xf62499ab927936ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbcf expref: 36 pid: 13908 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388506:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7348:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.388509:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7348:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.388512:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388519:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.388534:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1785220582.389485:0:9944:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 53 changes, 8 in progress, 4 in flight, 1374 done 00000100:00100000:0.0:1785220582.389503:0:14183:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d906f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_041:lustre-MDT0000-mdtlov_UUID+13:3653:x1871938510793600:12345-0@lo:6:osp-syn-1-0.0 Request processed in 25520us (25575us total) trans 4295070241 rc 0/0 00000100:00100000:0.0:1785220582.389512:0:14183:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284834 00010000:00010000:0.0:1785220582.389540:0:14261:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389551:0:14261:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.389560:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a260380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510795648:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.389560:0:14261:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389567:0:14261:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389582:0:14261:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:1.0:1785220582.389594:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.389622:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510795648 00000100:00100000:1.0:1785220582.389627:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510795648 00000100:00100000:1.0:1785220582.389658:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284841 00000100:00100000:1.0:1785220582.389665:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a261500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+14:3654:x1871938510795648:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.389686:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29477 count 1 00000100:00100000:0.0:1785220582.389692:0:14261:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d906fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_084:lustre-MDT0000-mdtlov_UUID+14:3653:x1871938510792320:12345-0@lo:6:osp-syn-1-0.0 Request processed in 27475us (27841us total) trans 4295070242 rc 0/0 00000100:00100000:0.0:1785220582.389698:0:14261:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284832 00010000:00010000:1.0:1785220582.389705:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389717:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.389720:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003185859c/0xf62499ab92793507 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ec expref: 67 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389726:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.389728:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7325:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.389733:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7325:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.389737:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389741:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389759:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.389773:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:1.0:1785220582.389774:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003185859c/0xf62499ab92793507 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ec expref: 67 pid: 14222 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.389785:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003185859c/0xf62499ab92793507 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ec expref: 67 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.389796:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a260000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510795904:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.389838:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.389859:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.389864:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a262d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510795776:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.389868:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a263800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+196:3655:x1871938510793856:12345-0@lo:6:osp-syn-0-0.0 Request processed in 25045us (25208us total) trans 4295070819 rc 0/0 00000100:00100000:0.0:1785220582.389874:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284835 00000100:00100000:1.0:1785220582.389885:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.389944:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+67:2406:x1871938520005120:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1856us (3415us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.389952:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100764 00000100:00100000:2.0:1785220582.389994:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100767 00000100:00100000:0.0:1785220582.389997:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a260a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510795520:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.390001:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+66:2404:x1871938520004992:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.390009:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.390015:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ade06539/0xf62499ab927934cf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9b4 expref: 66 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.390020:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.390026:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ade06539/0xf62499ab927934cf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9b4 expref: 66 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390043:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ade06539/0xf62499ab927934cf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9b4 expref: 66 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390055:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7321:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.390058:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7321:0x0].0x0 since=0s v=(1 0) 00000004:00080000:0.0:1785220582.390059:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d906f9f80/2, rc 0, transno 4295070241 00010000:00010000:2.0:1785220582.390062:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.390066:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510793600:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.390072:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.390075:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d94d57100/2, rc 0, transno 4295070242 00000100:00100000:0.0:1785220582.390078:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d94d57100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510792320:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.390103:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.390109:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a262a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510795392:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.390114:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.390139:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+65:2404:x1871938520004992:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 138us (3343us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.390145:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100767 00000100:00100000:2.0:1785220582.390160:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100768 00000100:00100000:2.0:1785220582.390168:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+64:2407:x1871938520005376:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.390175:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.390181:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000094d2bc9e/0xf62499ab927934c1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9a6 expref: 64 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390191:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000094d2bc9e/0xf62499ab927934c1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9a6 expref: 64 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390203:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094d2bc9e/0xf62499ab927934c1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9a6 expref: 64 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390213:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7320:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.390215:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7320:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.390219:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390226:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390248:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.390266:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+63:2407:x1871938520005376:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 99us (3315us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.390272:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100768 00010000:00010000:2.0:1785220582.390301:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220582.390309:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390317:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.390321:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390330:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390342:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390349:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a048043/0xf62499ab927a5b1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390367:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1785220582.390372:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390384:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98da00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.390387:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390394:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390403:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390409:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003faf5632/0xf62499ab927a5b54 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7322:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390423:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.390426:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390433:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98c400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.390436:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390444:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390453:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390461:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000547c73bb/0xf62499ab927a5b46 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x731e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.390465:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.390476:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.390480:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390487:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98d400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.390489:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390496:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390505:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390512:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b973d699/0xf62499ab927a5b5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390525:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.390527:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.390529:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.390535:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.390545:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510795776 00000100:00100000:2.0:1785220582.390550:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510795776 00000100:00100000:2.0:1785220582.390571:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510795520 00000100:00100000:2.0:1785220582.390574:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510795520 00000100:00100000:2.0:1785220582.390601:0:14266:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510795392 00000100:00100000:2.0:1785220582.390604:0:14266:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510795392 00000100:00100000:2.0:1785220582.390625:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284842 00000100:00100000:2.0:1785220582.390631:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a261180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+16:3654:x1871938510795776:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.390645:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29478 count 1 00010000:00010000:2.0:1785220582.390659:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390671:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008547db20/0xf62499ab92793515 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9fa expref: 62 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390678:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7326:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.390681:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7326:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.390684:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390703:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008547db20/0xf62499ab92793515 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9fa expref: 62 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390712:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008547db20/0xf62499ab92793515 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9fa expref: 62 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.390721:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510796032:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.390765:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24fb40 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.390779:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390788:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390797:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390803:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390816:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.390824:0:14260:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284843 00000100:00100000:0.0:1785220582.390833:0:14260:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+16:3653:x1871938510795520:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.390845:0:14260:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29476 count 1 00010000:00010000:0.0:1785220582.390856:0:14260:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.390866:0:14260:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006739fb93/0xf62499ab927934f9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9de expref: 62 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.390873:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7324:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.390875:0:14260:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7324:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.390879:0:14260:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.390896:0:14260:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006739fb93/0xf62499ab927934f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9de expref: 62 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.390905:0:14260:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006739fb93/0xf62499ab927934f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9de expref: 62 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.390907:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d906fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+17:3655:x1871938510792832:12345-0@lo:6:osp-syn-1-0.0 Request processed in 28853us (29099us total) trans 4295070243 rc 0/0 00000100:00100000:2.0:1785220582.390914:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284831 00000100:00100000:0.0:1785220582.390915:0:14260:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a704000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510796160:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.390928:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284844 00000100:00100000:2.0:1785220582.390933:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+16:3653:x1871938510795392:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.390936:0:14260:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00002000:00080000:2.0:1785220582.390943:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29475 count 1 00010000:00010000:2.0:1785220582.390954:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390965:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000058d45fa3/0xf62499ab927934eb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d0 expref: 62 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390972:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7323:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.390974:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7323:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.390978:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.390998:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000058d45fa3/0xf62499ab927934eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d0 expref: 62 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.391007:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000058d45fa3/0xf62499ab927934eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d0 expref: 62 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.391017:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510796288:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.391055:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a4d69c0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.391069:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.391073:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.391080:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f98dc00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.391115:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.391125:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.391137:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.391144:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a47a86f/0xf62499ab927a5b3f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7320:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.391159:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.391161:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.391163:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.391334:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a263b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510796544:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.391362:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.391391:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.391397:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a261f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510796416:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.391416:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.391435:0:14168:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510796544 00000100:00100000:0.0:1785220582.391439:0:14168:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510796544 00000100:00100000:0.0:1785220582.391460:0:14168:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510796416 00000100:00100000:0.0:1785220582.391463:0:14168:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510796416 00000100:00100000:0.0:1785220582.391479:0:14168:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284845 00000100:00100000:0.0:1785220582.391486:0:14168:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+18:3653:x1871938510796544:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.391498:0:14168:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29480 count 1 00010000:00010000:0.0:1785220582.391509:0:14168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.391520:0:14168:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000df9794a6/0xf62499ab92793531 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa16 expref: 62 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.391527:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7328:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.391529:0:14168:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7328:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.391533:0:14168:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.391551:0:14168:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000df9794a6/0xf62499ab92793531 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa16 expref: 62 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.391560:0:14168:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000df9794a6/0xf62499ab92793531 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa16 expref: 62 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.391569:0:14168:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510796672:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.391603:0:14168:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.391618:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284846 00000100:00100000:1.0:1785220582.391627:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+18:3653:x1871938510796416:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.391641:0:14210:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29479 count 1 00010000:00010000:1.0:1785220582.391653:0:14210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.391665:0:14210:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000041941013/0xf62499ab92793523 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa08 expref: 62 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.391672:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7327:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.391675:0:14210:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7327:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.391679:0:14210:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.391696:0:14210:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000041941013/0xf62499ab92793523 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa08 expref: 62 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.391706:0:14210:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000041941013/0xf62499ab92793523 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa08 expref: 62 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.391716:0:14210:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a263100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510796800:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.391739:0:14210:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000004:00080000:0.0:1785220582.392237:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d906f8e00/2, rc 0, transno 4295070819 00000100:00100000:0.0:1785220582.392255:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510793856:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1785220582.392268:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d906f9880/2, rc 0, transno 4295070243 00000100:00100000:0.0:1785220582.392274:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906f9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510792832:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.392284:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a263480/2, rc 0, transno 4295070818 00000100:00100000:0.0:1785220582.392287:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a263480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510791680:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.392292:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.392300:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510796928:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.392331:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.392788:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220582.392799:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a260e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510797056:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.392827:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.392954:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510797312:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.392981:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.393006:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510796928 00000100:00100000:2.0:1785220582.393011:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510796928 00000100:00100000:2.0:1785220582.393039:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510797056 00000100:00100000:2.0:1785220582.393042:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510797056 00000100:00100000:2.0:1785220582.393066:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510797312 00000100:00100000:2.0:1785220582.393069:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510797312 00000100:00100000:2.0:1785220582.393107:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284847 00000100:00100000:2.0:1785220582.393121:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+197:3655:x1871938510796928:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.393139:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29518 count 1 00010000:00010000:2.0:1785220582.393158:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.393175:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000031fff0d3/0xf62499ab9279373e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc23 expref: 35 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.393180:0:14196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284848 00010000:00010000:2.0:1785220582.393183:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x734e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.393186:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734e:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.393190:0:14196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c549500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+19:3654:x1871938510797056:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.393191:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 4/0,1 mode: --/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1785220582.393202:0:14196:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29481 count 1 00010000:00010000:2.0:1785220582.393213:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000031fff0d3/0xf62499ab9279373e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc23 expref: 35 pid: 8440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.393215:0:14196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.393226:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000031fff0d3/0xf62499ab9279373e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc23 expref: 35 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.393226:0:14196:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007756eceb/0xf62499ab9279353f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa24 expref: 62 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.393234:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7329:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.393237:0:14196:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7329:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220582.393238:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510797440:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.393240:0:14196:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.393257:0:14196:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007756eceb/0xf62499ab9279353f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa24 expref: 62 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.393265:0:14196:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007756eceb/0xf62499ab9279353f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa24 expref: 62 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.393280:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8e24f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.393280:0:14196:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a704380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510797568:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.393295:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284849 00000100:00100000:2.0:1785220582.393302:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b00b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+197:3653:x1871938510797312:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.393302:0:14196:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00002000:00080000:2.0:1785220582.393314:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29520 count 1 00000100:00100000:1.0:1785220582.393367:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510797184:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.393395:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.393419:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.393432:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000618d21cb/0xf62499ab9279375a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc3f expref: 35 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.393438:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7350:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.393441:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7350:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.393444:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.393464:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000618d21cb/0xf62499ab9279375a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc3f expref: 35 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.393473:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000618d21cb/0xf62499ab9279375a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc3f expref: 35 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.393504:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510797696:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.393527:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.393675:0:14201:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510797184 00000100:00100000:1.0:1785220582.393679:0:14201:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510797184 00000100:00100000:1.0:1785220582.393715:0:14201:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284850 00000100:00100000:1.0:1785220582.393723:0:14201:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98ad8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+198:3652:x1871938510797184:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.393736:0:14201:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29519 count 1 00010000:00010000:1.0:1785220582.393757:0:14201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.393769:0:14201:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000084589db0/0xf62499ab9279374c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc31 expref: 35 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.393775:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x734f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.393778:0:14201:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.393782:0:14201:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.393802:0:14201:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000084589db0/0xf62499ab9279374c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc31 expref: 35 pid: 14198 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.393812:0:14201:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000084589db0/0xf62499ab9279374c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc31 expref: 35 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.393821:0:14201:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d958cf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510797824:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.393858:0:14201:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.394166:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.394174:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.394187:0:14221:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.394197:0:14221:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.394207:0:14221:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.394215:0:14221:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.394241:0:14221:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.394256:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.394263:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.394270:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.394274:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.394282:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.394299:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.394306:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84622b7/0xf62499ab927a5b4d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7321:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.394323:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.394325:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.394326:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.394435:0:14221:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a262300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_067:lustre-MDT0000-mdtlov_UUID+20:3652:x1871938510793344:12345-0@lo:6:osp-syn-1-0.0 Request processed in 31432us (31624us total) trans 4295070244 rc 0/0 00000100:00100000:2.0:1785220582.394443:0:14221:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284833 00000100:00100000:3.0:1785220582.394801:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+35:2404:x1871938520005248:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 6333us (8191us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.394811:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100766 00000004:00080000:1.0:1785220582.395420:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d906fa680/2, rc 0, transno 4295070244 00000100:00100000:1.0:1785220582.395438:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d906fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510793344:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.395671:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510797952:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.395717:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.397495:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510797952 00000100:00100000:2.0:1785220582.397507:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510797952 00000100:00100000:2.0:1785220582.397612:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284851 00000100:00100000:2.0:1785220582.397623:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d958cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+20:3654:x1871938510797952:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.397656:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29482 count 1 00010000:00010000:2.0:1785220582.397683:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.397705:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003c5b7033/0xf62499ab9279354d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa32 expref: 62 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.397714:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x732a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.397718:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.397723:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.397769:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003c5b7033/0xf62499ab9279354d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa32 expref: 62 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.397781:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003c5b7033/0xf62499ab9279354d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa32 expref: 62 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.397795:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510798080:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.399214:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.399579:0:14158:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_020:lustre-OST0000_UUID:14158:1871938510794240:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.399723:0:14251:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_077:lustre-OST0000_UUID:14251:1871938510794496:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.399738:0:14251:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.399788:0:14158:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.402082:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.402403:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.402582:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a260000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510795904:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.402623:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.402727:0:14210:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a263100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_066:lustre-OST0001_UUID:14210:1871938510796800:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.402739:0:14210:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.402933:0:14201:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d958cf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_057:lustre-OST0000_UUID:14201:1871938510797824:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.402946:0:14201:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.402979:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.403055:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520005504 00000100:00100000:0.0:1785220582.403062:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520005504 00010000:00010000:0.0:1785220582.403129:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009ffdc4a3/0xf62499ab927936f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbdd expref: 35 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.403160:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520005632 00000100:00100000:0.0:1785220582.403163:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520005632 00010000:00010000:0.0:1785220582.403177:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006bff9206/0xf62499ab92793706 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbeb expref: 36 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.403199:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100769 00000100:00100000:0.0:1785220582.403208:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97b39500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2406:x1871938520005504:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.403223:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.403229:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009ffdc4a3/0xf62499ab927936f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbdd expref: 36 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403239:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009ffdc4a3/0xf62499ab927936f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbdd expref: 36 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403260:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009ffdc4a3/0xf62499ab927936f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbdd expref: 36 pid: 13836 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403276:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7349:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.403280:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7349:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.403286:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403296:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403322:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.403367:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97b39500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+35:2406:x1871938520005504:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 160us (1292us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.403376:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100769 00000100:00100000:0.0:1785220582.403392:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100770 00000100:00100000:0.0:1785220582.403398:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97b38e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+34:2406:x1871938520005632:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.403405:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.403410:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006bff9206/0xf62499ab92793706 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbeb expref: 34 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.403419:0:14204:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_060:lustre-OST0000_UUID:14204:1871938510795136:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.403419:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006bff9206/0xf62499ab92793706 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbeb expref: 34 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403432:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bff9206/0xf62499ab92793706 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbeb expref: 34 pid: 14156 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403433:0:14204:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403442:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.403445:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.403449:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.403458:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510796032:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.403462:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403469:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403480:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.403490:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:13836:1871938510796288:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.403491:0:14251:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403499:0:14251:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403500:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403511:0:14251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403519:0:14251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.403521:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:13819:1871938510797440:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.403530:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403534:0:14251:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.403550:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0000_UUID:14207:1871938510797696:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.403559:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403586:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.403592:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403600:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.403603:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403612:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403630:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403637:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d55836b/0xf62499ab927a5b69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14251 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.403657:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.403659:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.403661:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.403727:0:14251:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_077:lustre-MDT0000-mdtlov_UUID+199:3652:x1871938510794368:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20263us (20318us total) trans 4295070820 rc 0/0 00000100:00100000:0.0:1785220582.403736:0:14251:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284837 00000004:00080000:2.0:1785220582.403765:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89eff480/2, rc 0, transno 4295070820 00000100:00100000:2.0:1785220582.403777:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510794368:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.403780:0:14256:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_079:lustre-OST0000_UUID:14256:1871938510795264:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.403793:0:14256:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.403816:0:14260:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_083:lustre-OST0001_UUID:14260:1871938510796160:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.403838:0:14260:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.403861:0:14168:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_029:lustre-OST0001_UUID:14168:1871938510796672:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.403875:0:14168:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.403891:0:14196:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_053:lustre-OST0001_UUID:14196:1871938510797568:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.403897:0:14196:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.403920:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97b38e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+33:2406:x1871938520005632:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 523us (1519us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.403925:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100770 00010000:00010000:0.0:1785220582.403938:0:14158:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403943:0:14158:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403948:0:14158:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403952:0:14158:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.403969:0:14158:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.403990:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.403995:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.404002:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.404005:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.404013:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.404023:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.404036:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0789cf9/0xf62499ab927a5b62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14158 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.404039:0:14158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_020:lustre-MDT0000-mdtlov_UUID+199:3654:x1871938510794112:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21636us (21746us total) trans 4295070821 rc 0/0 00000100:00100000:0.0:1785220582.404045:0:14158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284836 00010000:00010000:2.0:1785220582.404051:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.404052:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.404054:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.404776:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510798208:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.404878:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.404928:0:14161:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510798208 00000100:00100000:2.0:1785220582.404934:0:14161:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510798208 00000100:00100000:2.0:1785220582.404972:0:14161:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284852 00000100:00100000:2.0:1785220582.404981:0:14161:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+199:3655:x1871938510798208:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.405004:0:14161:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29521 count 1 00000100:00100000:1.0:1785220582.405021:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.405024:0:14161:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405039:0:14161:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000005dfd6f2/0xf62499ab92793768 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc4d expref: 32 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.405040:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.405047:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7351:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.405051:0:14161:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7351:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.405054:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.405056:0:14161:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.405069:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.405083:0:14161:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000005dfd6f2/0xf62499ab92793768 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc4d expref: 32 pid: 14267 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.405100:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.405164:0:14161:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000005dfd6f2/0xf62499ab92793768 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc4d expref: 32 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.405178:0:14161:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510798336:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.405225:0:14161:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.405239:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006144 00000100:00100000:2.0:1785220582.405244:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006144 00010000:00010000:2.0:1785220582.405264:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003185859c/0xf62499ab92793507 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ec expref: 63 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.405264:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006272 00000100:00100000:0.0:1785220582.405268:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006272 00000100:00100000:2.0:1785220582.405283:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006656 00010000:00010000:0.0:1785220582.405284:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006739fb93/0xf62499ab927934f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9de expref: 64 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.405285:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006656 00010000:00010000:2.0:1785220582.405296:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000041941013/0xf62499ab92793523 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa08 expref: 65 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.405303:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006784 00000100:00100000:0.0:1785220582.405305:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006784 00000100:00100000:2.0:1785220582.405310:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007168 00000100:00100000:2.0:1785220582.405312:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007168 00010000:00010000:0.0:1785220582.405316:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007756eceb/0xf62499ab9279353f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa24 expref: 66 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405322:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000084589db0/0xf62499ab9279374c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc31 expref: 33 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.405329:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520005888 00000100:00100000:0.0:1785220582.405332:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520005888 00000100:00100000:2.0:1785220582.405337:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100771 00000100:00100000:2.0:1785220582.405342:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+66:2407:x1871938520006144:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.405342:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f21416d4/0xf62499ab92793722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc07 expref: 34 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405351:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.405356:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003185859c/0xf62499ab92793507 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ec expref: 66 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.405357:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100772 00000100:00100000:0.0:1785220582.405363:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d958cf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+66:2407:x1871938520006272:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.405367:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003185859c/0xf62499ab92793507 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ec expref: 66 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405369:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.405374:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006739fb93/0xf62499ab927934f9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9de expref: 66 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405385:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003185859c/0xf62499ab92793507 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9ec expref: 66 pid: 14222 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405387:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006739fb93/0xf62499ab927934f9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9de expref: 66 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405398:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7325:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.405398:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006739fb93/0xf62499ab927934f9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9de expref: 65 pid: 14249 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405401:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7325:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.405406:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405407:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7324:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.405410:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7324:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.405415:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405415:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405423:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405437:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.405437:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.405457:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e10c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+64:2407:x1871938520006144:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (2477us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.405463:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100771 00000100:00100000:3.0:1785220582.405497:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100773 00000100:00100000:2.0:1785220582.405505:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100774 00000100:00100000:3.0:1785220582.405506:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a263100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+63:2407:x1871938520006656:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1785220582.405510:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a260000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+63:2407:x1871938520006784:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.405513:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.405516:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.405518:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000041941013/0xf62499ab92793523 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa08 expref: 63 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405520:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007756eceb/0xf62499ab9279353f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa24 expref: 63 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.405529:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000041941013/0xf62499ab92793523 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa08 expref: 63 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405530:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007756eceb/0xf62499ab9279353f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa24 expref: 63 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.405542:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000041941013/0xf62499ab92793523 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa08 expref: 63 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405547:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007756eceb/0xf62499ab9279353f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa24 expref: 63 pid: 14186 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.405551:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7327:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.405554:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7327:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.405555:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7329:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.405557:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7329:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.405559:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405562:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.405567:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405569:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.405587:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.405589:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.405630:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d958cf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+61:2407:x1871938520006272:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 266us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.405638:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100772 00000100:00100000:0.0:1785220582.405654:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100775 00000100:00100000:0.0:1785220582.405661:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+34:2407:x1871938520007168:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.405669:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.405674:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000084589db0/0xf62499ab9279374c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc31 expref: 34 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405687:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000084589db0/0xf62499ab9279374c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc31 expref: 34 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405700:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000084589db0/0xf62499ab9279374c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc31 expref: 34 pid: 14198 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405709:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.405712:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.405716:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405722:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405738:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.405768:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+33:2407:x1871938520007168:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (697us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.405774:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100775 00000100:00100000:0.0:1785220582.405787:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100776 00000100:00100000:0.0:1785220582.405794:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+32:2407:x1871938520005888:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.405801:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.405805:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f21416d4/0xf62499ab92793722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc07 expref: 32 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405830:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f21416d4/0xf62499ab92793722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc07 expref: 32 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405842:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f21416d4/0xf62499ab92793722 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc07 expref: 32 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405854:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.405857:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.405860:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405867:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405884:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.405901:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+31:2407:x1871938520005888:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 108us (815us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.405907:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100776 00010000:00010000:0.0:1785220582.405936:0:14260:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405945:0:14260:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405954:0:14260:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405962:0:14260:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.405978:0:14260:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.406333:0:14260:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_083:lustre-MDT0000-mdtlov_UUID+21:3653:x1871938510795520:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15502us (16311us total) trans 4295070245 rc 0/0 00000100:00100000:0.0:1785220582.406341:0:14260:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284843 00010000:00010000:0.0:1785220582.406359:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.406364:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406372:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.406376:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406384:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406395:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406401:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5b93 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7324:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14260 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.406444:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a260000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+60:2407:x1871938520006784:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 932us (1387us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.406451:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100774 00000100:00100000:3.0:1785220582.406472:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a263100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+59:2407:x1871938520006656:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 968us (1430us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.406479:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100773 00010000:00010000:2.0:1785220582.406485:0:14204:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406494:0:14204:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406502:0:14204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406509:0:14204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406531:0:14204:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.406534:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.406538:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406543:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.406546:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406554:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406564:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406571:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000005ee637/0xf62499ab927a5b77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14204 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.406583:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.406585:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.406587:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.406615:0:14204:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_060:lustre-MDT0000-mdtlov_UUID+200:3654:x1871938510794752:12345-0@lo:6:osp-syn-0-0.0 Request processed in 20345us (22184us total) trans 4295070822 rc 0/0 00000100:00100000:2.0:1785220582.406621:0:14204:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284839 00010000:00010000:2.0:1785220582.406641:0:14196:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406651:0:14196:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406660:0:14196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406668:0:14196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406682:0:14196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.406768:0:14196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c549500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_053:lustre-MDT0000-mdtlov_UUID+21:3654:x1871938510797056:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13580us (13939us total) trans 4295070246 rc 0/0 00000100:00100000:2.0:1785220582.406775:0:14196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284848 00010000:00010000:2.0:1785220582.406792:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.406797:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406805:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.406808:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406815:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406827:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406834:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5bb6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7329:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.406849:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.406850:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.406852:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.406893:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a260a80/2, rc 0, transno 4295070245 00000100:00100000:2.0:1785220582.406906:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a260a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510795520:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.406964:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220582.406974:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510798464:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.407004:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.407134:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.407228:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.409204:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.409220:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.409231:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.409240:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.409269:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.409440:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a261500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+21:3654:x1871938510795648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19772us (19842us total) trans 4295070247 rc 0/0 00000100:00100000:3.0:1785220582.409449:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284841 00010000:00010000:2.0:1785220582.409489:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.409496:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409504:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.409508:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.409513:0:14210:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409522:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.409528:0:14210:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.409534:0:14210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.409540:0:14210:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409542:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409549:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5b85 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7325:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.409555:0:14210:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.409569:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.409572:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409579:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.409582:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409589:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409598:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409605:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093f9ae2c/0xf62499ab927a5ba8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7327:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14210 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.409624:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.409626:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.409628:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.409713:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:lustre-MDT0000-mdtlov_UUID+21:3653:x1871938510796416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 18087us (18294us total) trans 4295070248 rc 0/0 00000100:00100000:1.0:1785220582.409720:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284846 00000100:00100000:0.0:1785220582.409759:0:14178:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510798464 00000004:00080000:2.0:1785220582.409760:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a261f80/2, rc 0, transno 4295070248 00000100:00100000:0.0:1785220582.409766:0:14178:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510798464 00000100:00100000:2.0:1785220582.409771:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a261f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510796416:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.409810:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284853 00000100:00100000:2.0:1785220582.409831:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510798592:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.409865:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.409889:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+21:3653:x1871938510798464:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.409916:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29483 count 1 00010000:00010000:0.0:1785220582.409938:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.409955:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000086b76d18/0xf62499ab9279355b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa40 expref: 58 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.409964:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x732b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.409969:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.409973:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.409999:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000086b76d18/0xf62499ab9279355b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa40 expref: 58 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410011:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000086b76d18/0xf62499ab9279355b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa40 expref: 58 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.410022:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a704380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510798720:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.410077:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.410112:0:14160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510798592 00000100:00100000:0.0:1785220582.410116:0:14160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510798592 00000100:00100000:0.0:1785220582.410140:0:14160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284854 00000100:00100000:0.0:1785220582.410147:0:14160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+22:3653:x1871938510798592:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.410158:0:14160:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29484 count 1 00010000:00010000:0.0:1785220582.410170:0:14160:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410182:0:14160:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e3fbc9f6/0xf62499ab92793569 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa4e expref: 58 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410194:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x732c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.410198:0:14160:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.410202:0:14160:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410220:0:14160:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e3fbc9f6/0xf62499ab92793569 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa4e expref: 58 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410230:0:14160:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e3fbc9f6/0xf62499ab92793569 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa4e expref: 58 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.410240:0:14160:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510798848:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.410269:0:14160:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.410294:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520005760 00000100:00100000:0.0:1785220582.410298:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520005760 00010000:00010000:0.0:1785220582.410317:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000be509311/0xf62499ab92793730 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc15 expref: 31 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.410513:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100777 00000100:00100000:0.0:1785220582.410522:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+31:2407:x1871938520005760:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.410534:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.410540:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000be509311/0xf62499ab92793730 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc15 expref: 31 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410558:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000be509311/0xf62499ab92793730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc15 expref: 31 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410573:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be509311/0xf62499ab92793730 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc15 expref: 31 pid: 14260 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410585:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.410588:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.410594:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410602:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410678:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.410690:0:14256:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410696:0:14256:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410701:0:14256:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410705:0:14256:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.410719:0:14256:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.410740:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.410757:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.410765:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.410768:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.410775:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.410787:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.410794:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fda89c91/0xf62499ab927a5b7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14256 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.410811:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.410813:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.410815:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.410883:0:14256:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_079:lustre-MDT0000-mdtlov_UUID+200:3655:x1871938510794880:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24492us (26430us total) trans 4295070823 rc 0/0 00000100:00100000:0.0:1785220582.410890:0:14256:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284840 00000004:00080000:2.0:1785220582.410906:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efca80/2, rc 0, transno 4295070823 00000100:00100000:2.0:1785220582.410917:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510794880:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.410924:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+30:2407:x1871938520005760:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 404us (3697us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.410930:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100777 00000100:00100000:2.0:1785220582.410952:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220582.410960:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510798976:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.410976:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.410987:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.411034:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.411052:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006016 00000100:00100000:0.0:1785220582.411055:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006016 00010000:00010000:0.0:1785220582.411066:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008547db20/0xf62499ab92793515 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9fa expref: 59 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.411105:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006400 00000100:00100000:0.0:1785220582.411109:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006400 00010000:00010000:0.0:1785220582.411121:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000058d45fa3/0xf62499ab927934eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d0 expref: 60 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.411141:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100778 00000100:00100000:0.0:1785220582.411148:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac542850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+60:2407:x1871938520006016:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.411157:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.411162:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008547db20/0xf62499ab92793515 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9fa expref: 60 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411175:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008547db20/0xf62499ab92793515 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9fa expref: 60 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411188:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008547db20/0xf62499ab92793515 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9fa expref: 60 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411199:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7326:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.411202:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7326:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.411206:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411213:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411232:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.411251:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac542850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+59:2407:x1871938520006016:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.411257:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100778 00000100:00100000:0.0:1785220582.411295:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100779 00000100:00100000:0.0:1785220582.411302:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+58:2407:x1871938520006400:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.411309:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.411314:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000058d45fa3/0xf62499ab927934eb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d0 expref: 58 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411329:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000058d45fa3/0xf62499ab927934eb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d0 expref: 58 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411342:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000058d45fa3/0xf62499ab927934eb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7f9d0 expref: 58 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411352:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7323:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.411355:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7323:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.411359:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411366:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411384:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.411396:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411404:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411412:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411420:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411436:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.411443:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.411448:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.411456:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.411459:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.411466:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.411476:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.411483:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5b9a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7323:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.411497:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.411499:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.411501:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.411530:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+23:3653:x1871938510795392:12345-0@lo:6:osp-syn-1-0.0 Request processed in 20596us (21064us total) trans 4295070249 rc 0/0 00000100:00100000:0.0:1785220582.411537:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284844 00000004:00080000:2.0:1785220582.411543:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a262a00/2, rc 0, transno 4295070249 00000100:00100000:0.0:1785220582.411550:0:13836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510798976 00000100:00100000:2.0:1785220582.411551:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a262a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510795392:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.411552:0:13836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510798976 00000100:00100000:0.0:1785220582.411574:0:13836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284855 00000100:00100000:0.0:1785220582.411580:0:13836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b0e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+200:3655:x1871938510798976:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.411589:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510799104:0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.411596:0:13836:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29522 count 1 00000100:00100000:2.0:1785220582.411615:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.411794:0:13836:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411809:0:13836:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab20cc0a/0xf62499ab92793776 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc5b expref: 29 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411815:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7352:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.411818:0:13836:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7352:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.411821:0:13836:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411841:0:13836:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ab20cc0a/0xf62499ab92793776 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc5b expref: 29 pid: 14262 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411852:0:13836:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ab20cc0a/0xf62499ab92793776 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc5b expref: 29 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.411853:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510799104 00000100:00100000:3.0:1785220582.411858:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510799104 00000100:00100000:0.0:1785220582.411869:0:13836:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510799232:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.411882:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284856 00000100:00100000:3.0:1785220582.411888:0:14257:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97b39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+23:3655:x1871938510799104:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.411902:0:14257:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29485 count 1 00000100:00100000:0.0:1785220582.411902:0:13836:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.411914:0:14257:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411923:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.411926:0:14257:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000afaf6fa8/0xf62499ab92793577 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa5c expref: 57 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411932:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.411933:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x732d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.411936:0:14257:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.411940:0:14257:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411941:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411949:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.411959:0:14257:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000afaf6fa8/0xf62499ab92793577 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa5c expref: 57 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.411968:0:14257:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000afaf6fa8/0xf62499ab92793577 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa5c expref: 57 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.411970:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.411978:0:14257:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89eff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510799360:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.412002:0:14257:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38cc00 going to sleep for 11 seconds 00000004:00080000:3.0:1785220582.412039:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a260e00/2, rc 0, transno 4295070246 00000100:00100000:3.0:1785220582.412047:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a260e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510797056:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.412052:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a261180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+24:3654:x1871938510795776:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21421us (22166us total) trans 4295070250 rc 0/0 00000004:00080000:3.0:1785220582.412057:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a262d80/2, rc 0, transno 4295070250 00000100:00100000:0.0:1785220582.412059:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284842 00000100:00100000:3.0:1785220582.412060:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a262d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510795776:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.412068:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a260380/2, rc 0, transno 4295070247 00000100:00100000:3.0:1785220582.412073:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a260380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510795648:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.412099:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efea00/2, rc 0, transno 4295070822 00000100:00100000:3.0:1785220582.412106:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510794752:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1785220582.412213:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+57:2407:x1871938520006400:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 911us (1177us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.412221:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100779 00000100:00100000:0.0:1785220582.412282:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.412315:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.412351:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.412402:0:8442:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8442:1871938510795008:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.412422:0:8442:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.412462:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.412552:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.412618:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.412690:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006528 00000100:00100000:2.0:1785220582.412695:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006528 00010000:00010000:2.0:1785220582.412717:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000df9794a6/0xf62499ab92793531 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa16 expref: 57 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.412755:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520006912 00000100:00100000:2.0:1785220582.412759:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520006912 00010000:00010000:2.0:1785220582.412772:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000031fff0d3/0xf62499ab9279373e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc23 expref: 30 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.412788:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007040 00000100:00100000:2.0:1785220582.412791:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007040 00010000:00010000:2.0:1785220582.412803:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000618d21cb/0xf62499ab9279375a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc3f expref: 31 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.412821:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007296 00000100:00100000:2.0:1785220582.412824:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007296 00010000:00010000:2.0:1785220582.412834:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f02ec828/0xf62499ab92793714 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf9 expref: 32 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.412850:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007424 00000100:00100000:2.0:1785220582.412853:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007424 00010000:00010000:2.0:1785220582.412864:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003c5b7033/0xf62499ab9279354d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa32 expref: 58 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.412885:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007552 00000100:00100000:2.0:1785220582.412887:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007552 00010000:00010000:2.0:1785220582.412900:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000005dfd6f2/0xf62499ab92793768 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc4d expref: 33 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.412921:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100780 00000100:00100000:2.0:1785220582.412928:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+58:2407:x1871938520006528:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.412940:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.412945:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000df9794a6/0xf62499ab92793531 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa16 expref: 58 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.412957:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000df9794a6/0xf62499ab92793531 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa16 expref: 58 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.412977:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df9794a6/0xf62499ab92793531 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa16 expref: 58 pid: 14194 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.412992:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7328:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.412995:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7328:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413000:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413009:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413032:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.413060:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+57:2407:x1871938520006528:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 132us (777us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.413066:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100780 00000100:00100000:2.0:1785220582.413082:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100781 00000100:00100000:2.0:1785220582.413112:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+33:2407:x1871938520006912:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.413121:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.413126:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000031fff0d3/0xf62499ab9279373e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc23 expref: 33 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413136:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000031fff0d3/0xf62499ab9279373e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc23 expref: 33 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413151:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000031fff0d3/0xf62499ab9279373e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc23 expref: 33 pid: 8440 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413161:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413163:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413167:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413173:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413195:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.413215:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+32:2407:x1871938520006912:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (899us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.413223:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100781 00000100:00100000:2.0:1785220582.413236:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100782 00000100:00100000:2.0:1785220582.413242:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+31:2407:x1871938520007040:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.413249:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.413254:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000618d21cb/0xf62499ab9279375a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc3f expref: 31 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413262:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000618d21cb/0xf62499ab9279375a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc3f expref: 31 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413275:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000618d21cb/0xf62499ab9279375a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc3f expref: 31 pid: 14149 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413283:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7350:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413286:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7350:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413290:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413297:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413318:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.413337:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+30:2407:x1871938520007040:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (985us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.413344:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100782 00000100:00100000:2.0:1785220582.413356:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100783 00000100:00100000:2.0:1785220582.413362:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+29:2407:x1871938520007296:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.413370:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.413374:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f02ec828/0xf62499ab92793714 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf9 expref: 29 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413384:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f02ec828/0xf62499ab92793714 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf9 expref: 29 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413396:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f02ec828/0xf62499ab92793714 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf9 expref: 29 pid: 14178 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413405:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413408:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x734b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413412:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413418:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413436:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.413453:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+28:2407:x1871938520007296:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (990us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.413459:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100783 00000100:00100000:2.0:1785220582.413473:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100784 00000100:00100000:2.0:1785220582.413480:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+56:2407:x1871938520007424:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.413487:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.413492:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003c5b7033/0xf62499ab9279354d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa32 expref: 56 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413502:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003c5b7033/0xf62499ab9279354d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa32 expref: 56 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413515:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413518:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.413523:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413531:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.413543:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.413561:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+56:2407:x1871938520007424:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 81us (1009us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.413570:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100784 00000100:00100000:2.0:1785220582.413584:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100785 00000100:00100000:2.0:1785220582.413590:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+27:2406:x1871938520007552:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.413736:0:14168:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.413755:0:14168:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.413767:0:14168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.413774:0:14168:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.413791:0:14168:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.413945:0:14168:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_029:lustre-MDT0000-mdtlov_UUID+24:3653:x1871938510796544:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22457us (22579us total) trans 4295070251 rc 0/0 00000100:00100000:0.0:1785220582.413952:0:14168:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284845 00010000:00010000:0.0:1785220582.413979:0:8442:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.413989:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.413998:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414006:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414021:0:8442:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.414128:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+201:3652:x1871938510794624:12345-0@lo:6:osp-syn-0-0.0 Request processed in 28181us (29822us total) trans 4295070824 rc 0/0 00000100:00100000:0.0:1785220582.414135:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284838 00010000:00010000:0.0:1785220582.414154:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220582.414160:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414169:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.414173:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414181:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414194:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414202:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b84298a7/0xf62499ab927a5b8c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7326:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414222:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.414226:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414233:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.414236:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414244:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414254:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414261:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5ba1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7328:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414275:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.414278:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414284:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.414288:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414300:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414309:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414316:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004765897f/0xf62499ab927a5b70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.414329:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.414331:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.414332:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.414810:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.414819:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000005dfd6f2/0xf62499ab92793768 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc4d expref: 27 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.414831:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000005dfd6f2/0xf62499ab92793768 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc4d expref: 27 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.414846:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7351:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.414849:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7351:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.414854:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.414862:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.414877:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.414918:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+27:2406:x1871938520007552:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1327us (2298us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.414926:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100785 00010000:00010000:2.0:1785220582.415002:0:14172:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c5b7033/0xf62499ab9279354d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa32 expref: 55 pid: 14256 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.415022:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510798080:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.415043:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415053:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415060:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415069:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415076:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415109:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.415220:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d958cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+24:3654:x1871938510797952:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17599us (19502us total) trans 4295070252 rc 0/0 00000100:00100000:2.0:1785220582.415227:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284851 00010000:00010000:2.0:1785220582.415257:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415266:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415273:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415281:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.415289:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220582.415655:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.415660:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415668:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.415672:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415680:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415692:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415704:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eefecec7/0xf62499ab927a5bcb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415720:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.415723:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415729:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.415732:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.415733:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b008700/2, rc 0, transno 4295070821 00010000:00010000:0.0:1785220582.415739:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.415740:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510794112:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1785220582.415760:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415768:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000823c5b05/0xf62499ab927a5bbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.415782:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.415784:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00aa00/2, rc 0, transno 4295070252 00010000:00010000:0.0:1785220582.415784:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.415785:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.415789:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510797952:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.415858:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510798720:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.415873:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.415936:0:14160:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_022:lustre-OST0001_UUID:14160:1871938510798848:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.415949:0:14160:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.415994:0:13836:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:13836:1871938510799232:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.416006:0:13836:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416176:0:14201:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416186:0:14201:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416195:0:14201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416203:0:14201:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416216:0:14201:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.416310:0:14201:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98ad8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_057:lustre-MDT0000-mdtlov_UUID+201:3652:x1871938510797184:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22587us (22914us total) trans 4295070825 rc 0/0 00000100:00100000:0.0:1785220582.416320:0:14201:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284850 00010000:00010000:0.0:1785220582.416341:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.416346:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416354:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.416358:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416365:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416376:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416384:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5b90b3c/0xf62499ab927a5bc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.416398:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.416400:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.416401:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.416406:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.416428:0:14257:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89eff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_080:lustre-OST0001_UUID:14257:1871938510799360:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.416441:0:14257:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.416591:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b00b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+201:3653:x1871938510797312:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23289us (23609us total) trans 4295070826 rc 0/0 00000100:00100000:2.0:1785220582.416599:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284849 00010000:00010000:2.0:1785220582.416622:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416631:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416640:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416648:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416669:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.416758:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+201:3655:x1871938510796928:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23643us (24426us total) trans 4295070827 rc 0/0 00000100:00100000:2.0:1785220582.416767:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284847 00000004:00080000:3.0:1785220582.416773:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a704a80/2, rc 0, transno 4295070827 00000100:00100000:3.0:1785220582.416778:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510796928:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.416858:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.416863:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.416866:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93608700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510799744:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1785220582.416871:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.416874:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416881:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416893:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 1/0,0 mode: --/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.416899:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.416900:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2afd51f/0xf62499ab927a5baf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416913:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.416915:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.416916:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.416922:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.416944:0:14161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005dfd6f2/0xf62499ab92793768 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc4d expref: 26 pid: 14267 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.416954:0:14161:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_023:lustre-OST0000_UUID:14161:1871938510798336:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.416967:0:14161:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416975:0:14161:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416982:0:14161:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.416990:0:14161:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.417002:0:14161:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.417013:0:14161:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.417081:0:14161:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89efc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+201:3655:x1871938510798208:12345-0@lo:6:osp-syn-0-0.0 Request processed in 12101us (12202us total) trans 4295070828 rc 0/0 00000100:00100000:0.0:1785220582.417179:0:14174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510799744 00000100:00100000:0.0:1785220582.417185:0:14174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510799744 00000100:00100000:0.0:1785220582.417220:0:14174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284857 00000100:00100000:0.0:1785220582.417230:0:14174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d89eff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+202:3654:x1871938510799744:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1785220582.417253:0:14174:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29525 count 1 00010000:00010000:0.0:1785220582.417272:0:14174:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417286:0:14174:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004cbfc697/0xf62499ab927937a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc85 expref: 25 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417294:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7355:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.417299:0:14174:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7355:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.417304:0:14174:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.417317:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d310e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510800256:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.417325:0:14174:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004cbfc697/0xf62499ab927937a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc85 expref: 25 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417337:0:14174:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004cbfc697/0xf62499ab927937a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc85 expref: 25 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.417340:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.417349:0:14174:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510800384:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.417360:0:14227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510800256 00000100:00100000:3.0:1785220582.417363:0:14227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510800256 00000100:00100000:3.0:1785220582.417386:0:14227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284858 00000100:00100000:3.0:1785220582.417392:0:14227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d313b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+24:3654:x1871938510800256:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.417395:0:14174:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00002000:00080000:3.0:1785220582.417403:0:14227:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29489 count 1 00010000:00010000:3.0:1785220582.417417:0:14227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417421:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.417425:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417432:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.417434:0:14227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000048138d27/0xf62499ab927935af lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa94 expref: 54 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417436:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.417441:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7331:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.417444:0:14227:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7331:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.417445:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.417448:0:14227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417456:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417463:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab24a30d/0xf62499ab927a5bd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.417465:0:14227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000048138d27/0xf62499ab927935af lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa94 expref: 54 pid: 14159 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.417474:0:14227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000048138d27/0xf62499ab927935af lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa94 expref: 54 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.417478:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.417480:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.417481:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.417485:0:14227:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510800512:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.417486:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.417506:0:14227:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1785220582.417519:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220582.417525:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510799872:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.417546:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.417736:0:14223:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510799872 00000100:00100000:1.0:1785220582.417741:0:14223:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510799872 00000100:00100000:1.0:1785220582.417787:0:14223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284859 00000100:00100000:1.0:1785220582.417795:0:14223:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+25:3654:x1871938510799872:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.417808:0:14223:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29486 count 1 00000100:00100000:2.0:1785220582.417896:0:14161:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284852 00000100:00100000:2.0:1785220582.417921:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510799616:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.417946:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.417973:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d310380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510800128:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.417992:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.418045:0:14223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.418058:0:14223:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005a60277a/0xf62499ab92793585 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa6a expref: 54 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.418065:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x732e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.418068:0:14223:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.418072:0:14223:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.418121:0:14223:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005a60277a/0xf62499ab92793585 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa6a expref: 54 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.418131:0:14223:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005a60277a/0xf62499ab92793585 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa6a expref: 54 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.418140:0:14223:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a5ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0001_UUID:14223:1871938510800640:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.418171:0:14223:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.418238:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00a300/2, rc 0, transno 4295070826 00000100:00100000:2.0:1785220582.418249:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510797312:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.418298:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8a263b80/2, rc 0, transno 4295070251 00000100:00100000:2.0:1785220582.418307:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a263b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510796544:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.418335:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1785220582.418341:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97b38e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510799488:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.418364:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.422531:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d89efdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510800000:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.422608:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.422630:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510800896:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.422649:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.422703:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510799616 00000100:00100000:2.0:1785220582.422703:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93608e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510800768:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.422715:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510799616 00000100:00100000:2.0:1785220582.422732:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.422784:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510800128 00000100:00100000:3.0:1785220582.422788:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510800128 00000004:00080000:2.0:1785220582.422794:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b009c00/2, rc 0, transno 4295070828 00000100:00100000:2.0:1785220582.422806:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510798208:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.422809:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510799488 00000100:00100000:3.0:1785220582.422811:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510799488 00000100:00100000:3.0:1785220582.422826:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510800000 00000100:00100000:3.0:1785220582.422830:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510800000 00000100:00100000:3.0:1785220582.422846:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510800896 00000100:00100000:3.0:1785220582.422849:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510800896 00000100:00100000:3.0:1785220582.422863:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510800768 00000100:00100000:3.0:1785220582.422866:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510800768 00000100:00100000:3.0:1785220582.422888:0:14171:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284860 00000100:00100000:3.0:1785220582.422897:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93608a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+204:3653:x1871938510799616:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1785220582.422936:0:14171:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29524 count 1 00010000:00010000:3.0:1785220582.422967:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.422979:0:14193:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284861 00000100:00100000:2.0:1785220582.422988:0:14193:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+28:3653:x1871938510800128:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.422992:0:14171:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002edfeaf3/0xf62499ab92793792 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc77 expref: 25 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1785220582.423001:0:14193:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29488 count 1 00010000:00010000:3.0:1785220582.423003:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7354:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.423008:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7354:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.423013:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423019:0:14193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423031:0:14193:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d17445cd/0xf62499ab927935a1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa86 expref: 54 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423037:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7330:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.423039:0:14193:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7330:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.423042:0:14171:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002edfeaf3/0xf62499ab92793792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc77 expref: 25 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423044:0:14193:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.423056:0:14171:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002edfeaf3/0xf62499ab92793792 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc77 expref: 25 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423061:0:14193:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d17445cd/0xf62499ab927935a1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa86 expref: 54 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.423068:0:14171:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d313100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510801152:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.423069:0:14193:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d17445cd/0xf62499ab927935a1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa86 expref: 54 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.423079:0:14193:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0001_UUID:14193:1871938510801280:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.423150:0:14164:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284862 00000100:00100000:1.0:1785220582.423162:0:14164:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+204:3653:x1871938510799488:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.423177:0:14164:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29523 count 1 00010000:00010000:1.0:1785220582.423190:0:14164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423201:0:14164:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c5deda03/0xf62499ab92793784 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc69 expref: 25 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423208:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7353:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.423211:0:14164:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7353:0x0].0x0 since=0s v=(1 0) 00000100:00100000:3.0:1785220582.423214:0:14171:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c900 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.423214:0:14164:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423237:0:14164:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c5deda03/0xf62499ab92793784 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc69 expref: 25 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423246:0:14164:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c5deda03/0xf62499ab92793784 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc69 expref: 25 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.423256:0:14164:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a5ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510801408:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.423321:0:14197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284863 00000100:00100000:0.0:1785220582.423332:0:14197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+28:3653:x1871938510800000:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.423344:0:14197:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29487 count 1 00010000:00010000:0.0:1785220582.423358:0:14197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.423368:0:14197:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000028ee2db2/0xf62499ab92793593 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa78 expref: 54 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.423372:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x732f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.423374:0:14197:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.423376:0:14197:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.423389:0:14197:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000028ee2db2/0xf62499ab92793593 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa78 expref: 54 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.423395:0:14197:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000028ee2db2/0xf62499ab92793593 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa78 expref: 54 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.423401:0:14197:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510801536:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.423443:0:14193:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba8dfcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.423457:0:14163:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284864 00000100:00100000:2.0:1785220582.423461:0:14163:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+28:3653:x1871938510800896:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.423469:0:14163:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29490 count 1 00010000:00010000:2.0:1785220582.423478:0:14163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423485:0:14163:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000144f0f2f/0xf62499ab927935bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa2 expref: 54 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423489:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7332:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.423491:0:14163:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7332:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.423493:0:14163:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423504:0:14163:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000144f0f2f/0xf62499ab927935bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa2 expref: 54 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.423510:0:14163:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000144f0f2f/0xf62499ab927935bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa2 expref: 54 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.423516:0:14163:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0001_UUID:14163:1871938510801664:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.423530:0:14163:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00000004:00080000:2.0:1785220582.423582:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00ad80/2, rc 0, transno 4295070825 00000100:00100000:2.0:1785220582.423587:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510797184:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.423598:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efdf80/2, rc 0, transno 4295070824 00000100:00100000:2.0:1785220582.423601:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510794624:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.423609:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.423615:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510801024:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.423643:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.423708:0:14164:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.423724:0:14187:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510801024 00000100:00100000:1.0:1785220582.423728:0:14187:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510801024 00000100:00100000:1.0:1785220582.423796:0:14187:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284865 00000100:00100000:1.0:1785220582.423801:0:14187:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93608000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+205:3655:x1871938510800768:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1785220582.423810:0:14187:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29526 count 1 00010000:00010000:1.0:1785220582.423868:0:14187:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423877:0:14187:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bcaa44d5/0xf62499ab927937ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc93 expref: 25 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423882:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7356:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.423883:0:14187:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7356:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.423886:0:14187:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423899:0:14187:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bcaa44d5/0xf62499ab927937ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc93 expref: 25 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.423905:0:14187:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bcaa44d5/0xf62499ab927937ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc93 expref: 25 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.423912:0:14187:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a5efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510801792:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.423928:0:14187:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.423957:0:14197:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.423985:0:14250:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284866 00000100:00100000:2.0:1785220582.423990:0:14250:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+205:3652:x1871938510801024:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1785220582.423999:0:14250:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:29527 count 1 00010000:00010000:2.0:1785220582.424007:0:14250:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.424015:0:14250:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006a8661dc/0xf62499ab927937bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca1 expref: 25 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.424019:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x7357:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.424021:0:14250:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7357:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.424024:0:14250:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.424035:0:14250:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006a8661dc/0xf62499ab927937bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca1 expref: 25 pid: 14196 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.424041:0:14250:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006a8661dc/0xf62499ab927937bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca1 expref: 25 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.424046:0:14250:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510801920:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.424060:0:14250:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e82d19240 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.424377:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.424441:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.424474:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.424493:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007808 00000100:00100000:0.0:1785220582.424496:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007808 00010000:00010000:0.0:1785220582.424517:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e3fbc9f6/0xf62499ab92793569 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa4e expref: 55 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.424537:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008064 00000100:00100000:0.0:1785220582.424540:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008064 00010000:00010000:0.0:1785220582.424552:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000afaf6fa8/0xf62499ab92793577 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa5c expref: 56 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.424567:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007680 00000100:00100000:0.0:1785220582.424570:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007680 00010000:00010000:0.0:1785220582.424580:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000086b76d18/0xf62499ab9279355b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa40 expref: 57 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.424603:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100786 00000100:00100000:0.0:1785220582.424609:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+57:2407:x1871938520007808:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.424619:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.424624:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e3fbc9f6/0xf62499ab92793569 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa4e expref: 57 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424634:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e3fbc9f6/0xf62499ab92793569 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa4e expref: 57 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424653:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e3fbc9f6/0xf62499ab92793569 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa4e expref: 57 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424667:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.424670:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.424675:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424684:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.424702:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100787 00010000:00010000:0.0:1785220582.424706:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.424711:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+56:2407:x1871938520008064:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.424719:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.424724:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000afaf6fa8/0xf62499ab92793577 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa5c expref: 56 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424724:0:14160:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424735:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000afaf6fa8/0xf62499ab92793577 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa5c expref: 56 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424739:0:14160:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424759:0:14160:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424762:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000afaf6fa8/0xf62499ab92793577 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa5c expref: 56 pid: 14179 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424769:0:14160:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424773:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.424776:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.424780:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.424785:0:14160:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.424787:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424803:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.424823:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+55:2407:x1871938520008064:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 114us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.424830:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100787 00000100:00100000:3.0:1785220582.424852:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100788 00000100:00100000:3.0:1785220582.424860:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+54:2406:x1871938520007680:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.424869:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.424873:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000086b76d18/0xf62499ab9279355b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa40 expref: 54 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424882:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000086b76d18/0xf62499ab9279355b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa40 expref: 54 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424894:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000086b76d18/0xf62499ab9279355b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa40 expref: 54 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424903:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.424905:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.424909:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424915:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424931:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.424948:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+53:2406:x1871938520007680:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 90us (473us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.424954:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100788 00010000:00010000:3.0:1785220582.424972:0:14257:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424981:0:14257:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.424990:0:14257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.424994:0:14160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_022:lustre-MDT0000-mdtlov_UUID+29:3653:x1871938510798592:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14847us (15128us total) trans 4295070253 rc 0/0 00010000:00010000:3.0:1785220582.424997:0:14257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.425002:0:14160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284854 00010000:00010000:3.0:1785220582.425008:0:14257:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220582.425021:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.425027:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425035:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.425039:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425049:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425061:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425068:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5be0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14160 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.425080:0:14257:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97b39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_080:lustre-MDT0000-mdtlov_UUID+29:3655:x1871938510799104:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13193us (13465us total) trans 4295070254 rc 0/0 00010000:00010000:0.0:1785220582.425103:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.425107:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425114:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.425117:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425125:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425135:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425142:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a37a8de8/0xf62499ab927a5bee lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14257 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425155:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.425157:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.425159:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.425169:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425182:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425189:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425196:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425209:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.425281:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+30:3653:x1871938510798464:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15465us (18277us total) trans 4295070255 rc 0/0 00000100:00100000:3.0:1785220582.425282:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284856 00000100:00100000:0.0:1785220582.425287:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284853 00010000:00010000:0.0:1785220582.425302:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.425307:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425314:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.425318:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425325:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425333:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425338:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000457d4a7c/0xf62499ab927a5bd9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425349:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.425350:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.425351:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.425373:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+52:2407:x1871938520007808:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 765us (998us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.425377:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100786 00000100:00100000:0.0:1785220582.425418:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.425454:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520007936 00000100:00100000:3.0:1785220582.425459:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520007936 00010000:00010000:3.0:1785220582.425476:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ab20cc0a/0xf62499ab92793776 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc5b expref: 26 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.425482:0:14174:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_035:lustre-OST0000_UUID:14174:1871938510800384:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.425496:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100789 00010000:00010000:0.0:1785220582.425500:0:14174:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.425502:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+26:2406:x1871938520007936:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.425510:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.425515:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ab20cc0a/0xf62499ab92793776 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc5b expref: 26 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425524:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ab20cc0a/0xf62499ab92793776 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc5b expref: 26 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.425543:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.425544:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab20cc0a/0xf62499ab92793776 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc5b expref: 26 pid: 14262 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425555:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7352:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.425558:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7352:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.425562:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425570:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.425577:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:3.0:1785220582.425593:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.425677:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008192 00000100:00100000:2.0:1785220582.425681:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008192 00000100:00100000:3.0:1785220582.425682:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+25:2406:x1871938520007936:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 180us (263us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.425689:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100789 00010000:00010000:2.0:1785220582.425698:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004cbfc697/0xf62499ab927937a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc85 expref: 26 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.425701:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008320 00000100:00100000:3.0:1785220582.425703:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008320 00010000:00010000:3.0:1785220582.425714:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000048138d27/0xf62499ab927935af lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa94 expref: 52 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.425720:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100790 00000100:00100000:2.0:1785220582.425726:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+25:2407:x1871938520008192:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.425733:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100791 00010000:00010000:2.0:1785220582.425734:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1785220582.425738:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+52:2407:x1871938520008320:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.425738:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004cbfc697/0xf62499ab927937a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc85 expref: 25 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425744:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.425763:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000048138d27/0xf62499ab927935af lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa94 expref: 52 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.425763:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004cbfc697/0xf62499ab927937a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc85 expref: 25 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425773:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000048138d27/0xf62499ab927935af lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa94 expref: 52 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.425778:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cbfc697/0xf62499ab927937a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc85 expref: 25 pid: 14208 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425784:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7331:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.425787:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7331:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.425788:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7355:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.425791:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.425791:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7355:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.425795:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425799:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.425803:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425811:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.425825:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+52:2407:x1871938520008320:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 87us (247us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.425831:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100791 00010000:00010000:3.0:1785220582.425865:0:14227:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000048138d27/0xf62499ab927935af lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa94 expref: 51 pid: 14159 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.425876:0:14227:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_070:lustre-OST0001_UUID:14227:1871938510800512:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.425888:0:14227:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425896:0:14227:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425907:0:14227:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425917:0:14227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.425925:0:14227:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.425973:0:13836:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426005:0:13836:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426013:0:13836:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426025:0:13836:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.426052:0:14227:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.426053:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.426069:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+24:2407:x1871938520008192:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 342us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.426075:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100790 00010000:00010000:1.0:1785220582.426151:0:14174:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.426159:0:14174:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.426168:0:14174:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.426175:0:14174:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426219:0:13836:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.426318:0:13836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b0e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+206:3655:x1871938510798976:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14737us (15329us total) trans 4295070829 rc 0/0 00000100:00100000:3.0:1785220582.426319:0:14227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d313b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_070:lustre-MDT0000-mdtlov_UUID+29:3654:x1871938510800256:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8927us (8978us total) trans 4295070256 rc 0/0 00000100:00100000:0.0:1785220582.426325:0:13836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284855 00000100:00100000:3.0:1785220582.426327:0:14227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284858 00010000:00010000:0.0:1785220582.426415:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220582.426419:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426425:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.426427:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426433:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426441:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426445:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a1c24de/0xf62499ab927a5bfc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7331:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426457:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220582.426459:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426463:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.426465:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426469:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426474:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426478:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e74303e/0xf62499ab927a5be7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426486:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.426487:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426491:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.426493:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426497:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426505:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.426510:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.426510:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.426511:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.426515:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.426548:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.426585:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00a680/2, rc 0, transno 4295070254 00000100:00100000:2.0:1785220582.426598:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510799104:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.426611:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b009880/2, rc 0, transno 4295070829 00000100:00100000:2.0:1785220582.426615:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510798976:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1785220582.426638:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d310e00/2, rc 0, transno 4295070256 00000100:00100000:2.0:1785220582.426642:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d310e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510800256:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.426685:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00b800/2, rc 0, transno 4295070253 00000100:00100000:2.0:1785220582.426695:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510798592:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.426705:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00b100/2, rc 0, transno 4295070255 00000100:00100000:2.0:1785220582.426707:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510798464:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.426726:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510802048:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.426771:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.426803:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.426807:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510802176:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.426876:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.426897:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d313800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510802304:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.426910:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.426928:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.426933:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510802432:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.426943:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510802048 00000100:00100000:3.0:1785220582.426948:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510802048 00000100:00100000:2.0:1785220582.426952:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.426972:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510802176 00000100:00100000:3.0:1785220582.426975:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510802176 00000100:00100000:3.0:1785220582.427001:0:14208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510802304 00000100:00100000:3.0:1785220582.427004:0:14208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510802304 00000100:00100000:0.0:1785220582.427011:0:14186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510802432 00000100:00100000:0.0:1785220582.427014:0:14186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510802432 00000100:00100000:3.0:1785220582.427023:0:14208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284867 00000100:00100000:3.0:1785220582.427029:0:14208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93608380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+32:3654:x1871938510802048:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.427035:0:14186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284868 00000100:00100000:0.0:1785220582.427040:0:14186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+32:3654:x1871938510802176:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.427052:0:14208:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29491 count 1 00002000:00080000:0.0:1785220582.427052:0:14186:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29492 count 1 00010000:00010000:3.0:1785220582.427073:0:14208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427073:0:14186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427114:0:14186:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e706ad15/0xf62499ab927935d9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fabe expref: 50 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427124:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7334:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.427128:0:14186:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7334:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.427131:0:14186:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427154:0:14186:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e706ad15/0xf62499ab927935d9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fabe expref: 50 pid: 14227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427173:0:14186:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e706ad15/0xf62499ab927935d9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fabe expref: 50 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.427187:0:14186:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510802560:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.427221:0:14186:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.427238:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284869 00010000:00010000:3.0:1785220582.427243:0:14208:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000608b9ffa/0xf62499ab927935cb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab0 expref: 50 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.427244:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+32:3652:x1871938510802304:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.427252:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7333:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.427255:0:14208:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7333:0x0].0x0 since=0s v=(1 0) 00002000:00080000:0.0:1785220582.427256:0:14202:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29493 count 1 00010000:00010000:3.0:1785220582.427259:0:14208:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427267:0:14202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.427275:0:14208:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000608b9ffa/0xf62499ab927935cb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab0 expref: 50 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427278:0:14202:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fcf67aaa/0xf62499ab927935e7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7facc expref: 50 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.427285:0:14208:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000608b9ffa/0xf62499ab927935cb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab0 expref: 50 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.427285:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7335:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.427288:0:14202:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7335:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.427292:0:14202:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.427296:0:14208:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510802688:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.427307:0:14202:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fcf67aaa/0xf62499ab927935e7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7facc expref: 50 pid: 14259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.427314:0:14208:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c180 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.427316:0:14202:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fcf67aaa/0xf62499ab927935e7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7facc expref: 50 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.427329:0:14156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284870 00000100:00100000:0.0:1785220582.427330:0:14202:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a704700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510802816:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.427336:0:14156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+32:3652:x1871938510802432:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.427348:0:14156:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29494 count 1 00000100:00100000:0.0:1785220582.427348:0:14202:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.427359:0:14156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.427371:0:14156:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002b8c95ba/0xf62499ab927935f5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fada expref: 50 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.427378:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7336:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.427381:0:14156:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7336:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.427386:0:14156:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.427404:0:14156:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002b8c95ba/0xf62499ab927935f5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fada expref: 50 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.427413:0:14156:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002b8c95ba/0xf62499ab927935f5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fada expref: 50 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.427422:0:14156:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d311880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510802944:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.427441:0:14156:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c0c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.427478:0:14174:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1785220582.427484:0:14174:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3c906d4/0xf62499ab927a5bf5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14174 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.427602:0:14174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d89eff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_035:lustre-MDT0000-mdtlov_UUID+206:3654:x1871938510799744:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10374us (10703us total) trans 4295070830 rc 0/0 00000100:00100000:1.0:1785220582.427608:0:14174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284857 00010000:00010000:0.0:1785220582.427782:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.428025:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93608700/2, rc 0, transno 4295070830 00000100:00100000:2.0:1785220582.428031:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93608700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510799744:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1785220582.431303:0:14163:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_025:lustre-OST0001_UUID:14163:1871938510801664:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.431347:0:14163:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.432006:0:14223:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a5ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_069:lustre-OST0001_UUID:14223:1871938510800640:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.432007:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.432026:0:14223:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.432077:0:14193:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_050:lustre-OST0001_UUID:14193:1871938510801280:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.432214:0:14171:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d313100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_032:lustre-OST0000_UUID:14171:1871938510801152:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.432234:0:14171:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.432260:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008576 00000100:00100000:3.0:1785220582.432266:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008576 00010000:00010000:3.0:1785220582.432302:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002edfeaf3/0xf62499ab92793792 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc77 expref: 24 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.432536:0:14193:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.432707:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100792 00000100:00100000:0.0:1785220582.432721:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8c304450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+24:2407:x1871938520008576:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.432737:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.432744:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002edfeaf3/0xf62499ab92793792 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc77 expref: 24 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.432765:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002edfeaf3/0xf62499ab92793792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc77 expref: 24 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.432790:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002edfeaf3/0xf62499ab92793792 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc77 expref: 24 pid: 13820 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.432807:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7354:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.432811:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7354:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.432816:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.432826:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.432866:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.432908:0:14171:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.432918:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.432920:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8c304450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+23:2407:x1871938520008576:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 203us (914us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.432928:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100792 00010000:00010000:2.0:1785220582.432930:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.432937:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.432954:0:14171:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.432983:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.432988:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.432996:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.433000:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.433007:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.433020:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.433027:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000009e8b804/0xf62499ab927a5c0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.433045:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.433047:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.433048:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.433145:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_032:lustre-MDT0000-mdtlov_UUID+206:3653:x1871938510799616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10247us (15196us total) trans 4295070831 rc 0/0 00000100:00100000:2.0:1785220582.433152:0:14171:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284860 00000004:00080000:2.0:1785220582.433192:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b009180/2, rc 0, transno 4295070831 00000100:00100000:2.0:1785220582.433197:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510799616:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1785220582.433932:0:14164:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a5ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_026:lustre-OST0000_UUID:14164:1871938510801408:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.433950:0:14164:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.435269:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.435302:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.435319:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.435342:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008448 00000100:00100000:0.0:1785220582.435348:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008448 00010000:00010000:0.0:1785220582.435380:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005a60277a/0xf62499ab92793585 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa6a expref: 51 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.435407:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008704 00000100:00100000:0.0:1785220582.435410:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008704 00010000:00010000:0.0:1785220582.435423:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000144f0f2f/0xf62499ab927935bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa2 expref: 52 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.435445:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008960 00000100:00100000:0.0:1785220582.435448:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008960 00010000:00010000:0.0:1785220582.435461:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c5deda03/0xf62499ab92793784 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc69 expref: 23 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.435481:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100793 00000100:00100000:0.0:1785220582.435490:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+52:2406:x1871938520008448:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.435504:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.435510:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005a60277a/0xf62499ab92793585 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa6a expref: 52 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.435521:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005a60277a/0xf62499ab92793585 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa6a expref: 52 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.435543:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a60277a/0xf62499ab92793585 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa6a expref: 52 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.435561:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.435567:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732e:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220582.435663:0:14197:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_054:lustre-OST0001_UUID:14197:1871938510801536:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.435691:0:14197:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.435778:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100794 00000100:00100000:3.0:1785220582.435790:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+51:2406:x1871938520008704:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.435799:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.435804:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000144f0f2f/0xf62499ab927935bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa2 expref: 51 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.435821:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000144f0f2f/0xf62499ab927935bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa2 expref: 51 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.435835:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000144f0f2f/0xf62499ab927935bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faa2 expref: 51 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.435846:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7332:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.435849:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7332:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.435855:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.435865:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.435896:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.435939:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+50:2406:x1871938520008704:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 153us (636us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.435946:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100794 00000100:00100000:3.0:1785220582.435962:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100795 00000100:00100000:3.0:1785220582.435969:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+23:2406:x1871938520008960:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.435977:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.435981:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c5deda03/0xf62499ab92793784 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc69 expref: 23 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.435996:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c5deda03/0xf62499ab92793784 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc69 expref: 23 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.436010:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5deda03/0xf62499ab92793784 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc69 expref: 23 pid: 14206 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.436021:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7353:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.436025:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7353:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.436029:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.436036:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.436054:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.436072:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+22:2406:x1871938520008960:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 103us (752us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.436078:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100795 00000100:00100000:1.0:1785220582.436388:0:14187:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a5efb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_044:lustre-OST0000_UUID:14187:1871938510801792:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.436408:0:14187:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.436427:0:14164:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.436435:0:14164:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.436445:0:14164:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.436453:0:14164:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.436473:0:14164:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.436715:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.436724:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.436756:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.436778:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+49:2406:x1871938520008448:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1290us (1514us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.436786:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100793 00000100:00100000:0.0:1785220582.436846:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.436908:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.437012:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.437037:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.437290:0:14164:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_026:lustre-MDT0000-mdtlov_UUID+206:3653:x1871938510799488:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14131us (18923us total) trans 4295070832 rc 0/0 00000100:00100000:1.0:1785220582.437299:0:14164:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284862 00010000:00010000:1.0:1785220582.437326:0:14223:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437335:0:14223:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437344:0:14223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437350:0:14223:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437361:0:14223:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.437471:0:14223:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_069:lustre-MDT0000-mdtlov_UUID+33:3654:x1871938510799872:12345-0@lo:6:osp-syn-1-0.0 Request processed in 19677us (19923us total) trans 4295070257 rc 0/0 00000100:00100000:1.0:1785220582.437478:0:14223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284859 00010000:00010000:1.0:1785220582.437499:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.437506:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437514:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.437519:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437533:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437549:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437557:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000927db29/0xf62499ab927a5c18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14164 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437578:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.437582:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437588:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.437592:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.437599:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.437768:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009216 00000100:00100000:3.0:1785220582.437775:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009216 00010000:00010000:3.0:1785220582.437807:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000028ee2db2/0xf62499ab92793593 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa78 expref: 49 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.437853:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009088 00000100:00100000:2.0:1785220582.437857:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009088 00010000:00010000:2.0:1785220582.437876:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bcaa44d5/0xf62499ab927937ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc93 expref: 22 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.437896:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520008832 00000100:00100000:2.0:1785220582.437899:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520008832 00000100:00100000:3.0:1785220582.437909:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009344 00010000:00010000:2.0:1785220582.437911:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d17445cd/0xf62499ab927935a1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa86 expref: 50 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.437912:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009344 00010000:00010000:3.0:1785220582.437923:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006a8661dc/0xf62499ab927937bc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca1 expref: 23 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.437936:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100796 00000100:00100000:3.0:1785220582.437940:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100797 00000100:00100000:2.0:1785220582.437943:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+23:2406:x1871938520009088:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.437945:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+50:2406:x1871938520009216:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.437954:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.437954:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.437960:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000028ee2db2/0xf62499ab92793593 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa78 expref: 50 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.437960:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bcaa44d5/0xf62499ab927937ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc93 expref: 23 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.437972:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000028ee2db2/0xf62499ab92793593 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa78 expref: 50 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.437972:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bcaa44d5/0xf62499ab927937ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc93 expref: 23 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.437989:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000028ee2db2/0xf62499ab92793593 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa78 expref: 50 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.437989:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bcaa44d5/0xf62499ab927937ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc93 expref: 23 pid: 14202 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.438002:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.438002:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7356:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.438006:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x732f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.438006:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7356:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.438011:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438011:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.438021:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438021:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.438041:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.438041:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.438069:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+22:2406:x1871938520009088:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 126us (1160us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.438076:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100796 00010000:00010000:1.0:1785220582.438077:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438103:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c03 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438126:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.438128:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.438130:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.438139:0:14187:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438148:0:14187:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438157:0:14187:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438164:0:14187:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438178:0:14187:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.438267:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100798 00000100:00100000:2.0:1785220582.438275:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+49:2407:x1871938520008832:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.438283:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.438288:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d17445cd/0xf62499ab927935a1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa86 expref: 49 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.438291:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+49:2406:x1871938520009216:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 345us (1445us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.438299:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d17445cd/0xf62499ab927935a1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa86 expref: 49 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.438301:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100797 00010000:00010000:2.0:1785220582.438311:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d17445cd/0xf62499ab927935a1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fa86 expref: 48 pid: 14193 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.438315:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100799 00000100:00100000:3.0:1785220582.438322:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e8ce7b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+21:2406:x1871938520009344:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.438322:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7330:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.438325:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7330:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.438328:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.438329:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.438333:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006a8661dc/0xf62499ab927937bc lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca1 expref: 21 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438337:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.438343:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006a8661dc/0xf62499ab927937bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca1 expref: 21 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.438354:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7357:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.438354:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.438357:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7357:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.438361:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.438368:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.438372:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2407:x1871938520008832:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 98us (1360us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.438378:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100798 00010000:00010000:3.0:1785220582.438380:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.438395:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e8ce7b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+21:2406:x1871938520009344:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 74us (1357us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220582.438398:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1785220582.438401:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100799 00010000:00010000:2.0:1785220582.438404:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438411:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.438413:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438420:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438430:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438437:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ca24b/0xf62499ab927a5c2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14187 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438452:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.438453:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.438455:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.438498:0:14250:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a8661dc/0xf62499ab927937bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca1 expref: 20 pid: 14196 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.438507:0:14187:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_044:lustre-MDT0000-mdtlov_UUID+206:3655:x1871938510800768:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14705us (15774us total) trans 4295070833 rc 0/0 00000100:00100000:2.0:1785220582.438509:0:14250:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_076:lustre-OST0000_UUID:14250:1871938510801920:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.438514:0:14187:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284865 00010000:00010000:2.0:1785220582.438525:0:14250:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438531:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.438534:0:14250:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438540:0:14250:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438548:0:14250:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438554:0:14250:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438566:0:14250:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.438648:0:14250:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_076:lustre-MDT0000-mdtlov_UUID+206:3652:x1871938510801024:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14657us (15003us total) trans 4295070834 rc 0/0 00000100:00100000:2.0:1785220582.438655:0:14250:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284866 00010000:00010000:2.0:1785220582.438678:0:14163:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438686:0:14163:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438695:0:14163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438701:0:14163:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.438705:0:14208:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_064:lustre-OST0001_UUID:14208:1871938510802688:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.438718:0:14208:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438725:0:14163:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1785220582.438762:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93608e00/2, rc 0, transno 4295070833 00000100:00100000:3.0:1785220582.438769:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93608e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510800768:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1785220582.438816:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:2.0:1785220582.438818:0:14163:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_025:lustre-MDT0000-mdtlov_UUID+33:3653:x1871938510800896:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15356us (16167us total) trans 4295070258 rc 0/0 00010000:00010000:1.0:1785220582.438820:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.438825:0:14163:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284864 00010000:00010000:1.0:1785220582.438828:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.438831:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438839:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438844:0:14193:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438850:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438853:0:14193:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438858:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07f87fc/0xf62499ab927a5c34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14250 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438861:0:14193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438868:0:14193:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438870:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.438873:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438879:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.438882:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438888:0:14193:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.438890:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438898:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438905:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8347405/0xf62499ab927a5c26 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7332:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438923:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.438927:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438933:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.438935:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438943:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.438952:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.438954:0:14193:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_050:lustre-MDT0000-mdtlov_UUID+33:3653:x1871938510800128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 15968us (20961us total) trans 4295070259 rc 0/0 00010000:00010000:1.0:1785220582.438960:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095f9a9bf/0xf62499ab927a5c11 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7330:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.438961:0:14193:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284861 00010000:00010000:1.0:1785220582.438972:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.438974:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.438975:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.438980:0:14197:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.438989:0:14197:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.439002:0:14197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.439010:0:14197:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.439027:0:14197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.439080:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.439101:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439110:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.439113:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439121:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439133:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439140:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5c1f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x732f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14197 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439154:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.439156:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.439157:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.439177:0:14156:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_019:lustre-OST0001_UUID:14156:1871938510802944:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.439188:0:14156:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.439255:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.439279:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.439343:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.439404:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.439421:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009856 00000100:00100000:3.0:1785220582.439426:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009856 00010000:00010000:3.0:1785220582.439448:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fcf67aaa/0xf62499ab927935e7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7facc expref: 47 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.439473:0:14238:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009600 00000100:00100000:3.0:1785220582.439476:0:14238:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009600 00010000:00010000:3.0:1785220582.439487:0:14238:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000608b9ffa/0xf62499ab927935cb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab0 expref: 48 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.439506:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100800 00000100:00100000:3.0:1785220582.439512:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+48:2405:x1871938520009856:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.439523:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.439527:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fcf67aaa/0xf62499ab927935e7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7facc expref: 48 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439536:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fcf67aaa/0xf62499ab927935e7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7facc expref: 48 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439549:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7335:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.439553:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7335:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.439558:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439565:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439579:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.439601:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+48:2405:x1871938520009856:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 89us (322us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.439621:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100800 00000100:00100000:3.0:1785220582.439637:0:14238:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100801 00000100:00100000:3.0:1785220582.439644:0:14238:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2407:x1871938520009600:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.439652:0:14238:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.439657:0:14238:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000608b9ffa/0xf62499ab927935cb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab0 expref: 47 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439666:0:14238:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000608b9ffa/0xf62499ab927935cb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab0 expref: 47 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439678:0:14238:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000608b9ffa/0xf62499ab927935cb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fab0 expref: 47 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439684:0:14202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fcf67aaa/0xf62499ab927935e7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7facc expref: 47 pid: 14259 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439687:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7333:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.439690:0:14238:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7333:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.439694:0:14238:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.439696:0:14202:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a704700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_058:lustre-OST0001_UUID:14202:1871938510802816:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.439702:0:14238:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439710:0:14202:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439719:0:14202:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439722:0:14238:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.439727:0:14202:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439735:0:14202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.439738:0:14238:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5ed500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+45:2407:x1871938520009600:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (395us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220582.439743:0:14202:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.439744:0:14238:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100801 00010000:00010000:0.0:1785220582.439772:0:14202:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.439773:0:14208:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439782:0:14208:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439790:0:14208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439797:0:14208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439808:0:14208:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1785220582.439861:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:lustre-MDT0000-mdtlov_UUID+35:3652:x1871938510802304:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12617us (12948us total) trans 4295070261 rc 0/0 00000100:00100000:0.0:1785220582.439868:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284869 00000100:00100000:3.0:1785220582.439872:0:14208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_064:lustre-MDT0000-mdtlov_UUID+35:3654:x1871938510802048:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12844us (13102us total) trans 4295070262 rc 0/0 00000100:00100000:3.0:1785220582.439877:0:14208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284867 00010000:00010000:3.0:1785220582.439890:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.439893:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.439895:0:14186:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_043:lustre-OST0001_UUID:14186:1871938510802560:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.439901:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.439905:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439907:0:14186:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439914:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439921:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.439924:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439925:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439933:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e980d9e/0xf62499ab927a5c49 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7335:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439933:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.439936:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.439946:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.439947:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.439949:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.439949:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439959:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439965:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5c3b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7333:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.439979:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.439980:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.439982:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.440660:0:14197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_054:lustre-MDT0000-mdtlov_UUID+33:3653:x1871938510800000:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17328us (18048us total) trans 4295070260 rc 0/0 00000100:00100000:2.0:1785220582.440669:0:14197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284863 00000004:00080000:2.0:1785220582.440711:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d313800/2, rc 0, transno 4295070261 00000100:00100000:2.0:1785220582.440724:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d313800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510802304:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.440787:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220582.440800:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510803072:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.440801:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d311500/2, rc 0, transno 4295070262 00000100:00100000:3.0:1785220582.440815:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510802048:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.440827:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efc000/2, rc 0, transno 4295070257 00000100:00100000:3.0:1785220582.440836:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510799872:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.440841:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1785220582.440849:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b00b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510803200:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.440872:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220582.440885:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97b38e00/2, rc 0, transno 4295070832 00000100:00100000:3.0:1785220582.440890:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97b38e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510799488:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1785220582.440898:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d89efdc00/2, rc 0, transno 4295070260 00000100:00100000:0.0:1785220582.440957:0:14194:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510803072 00000100:00100000:0.0:1785220582.440963:0:14194:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510803072 00000100:00100000:0.0:1785220582.440995:0:14194:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284871 00000100:00100000:0.0:1785220582.441002:0:14194:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+33:3655:x1871938510803072:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.441025:0:14194:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29495 count 1 00010000:00010000:0.0:1785220582.441049:0:14194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441065:0:14194:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7a15f1d/0xf62499ab92793603 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae8 expref: 44 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441074:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7337:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.441077:0:14194:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7337:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.441082:0:14194:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.441127:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.441159:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.441496:0:14194:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e7a15f1d/0xf62499ab92793603 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae8 expref: 44 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441509:0:14194:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e7a15f1d/0xf62499ab92793603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae8 expref: 44 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.441522:0:14194:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510803456:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.441557:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d89efdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510800000:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.441579:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9360a300/2, rc 0, transno 4295070258 00000100:00100000:3.0:1785220582.441583:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510800896:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.441585:0:14194:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000004:00080000:3.0:1785220582.441592:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d310380/2, rc 0, transno 4295070259 00000100:00100000:3.0:1785220582.441595:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d310380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510800128:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.441603:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220582.441604:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009472 00000100:00100000:0.0:1785220582.441607:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009472 00010000:00010000:0.0:1785220582.441625:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e706ad15/0xf62499ab927935d9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fabe expref: 45 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.441646:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009728 00000100:00100000:0.0:1785220582.441649:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009728 00010000:00010000:0.0:1785220582.441661:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002b8c95ba/0xf62499ab927935f5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fada expref: 46 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.441682:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100802 00000100:00100000:0.0:1785220582.441689:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+46:2406:x1871938520009472:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.441697:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.441702:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e706ad15/0xf62499ab927935d9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fabe expref: 46 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441717:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e706ad15/0xf62499ab927935d9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fabe expref: 46 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441734:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e706ad15/0xf62499ab927935d9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fabe expref: 46 pid: 14227 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441757:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7334:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.441760:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7334:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.441765:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441773:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441797:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.441819:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+45:2406:x1871938520009472:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 131us (2414us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.441825:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100802 00000100:00100000:0.0:1785220582.441840:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100803 00000100:00100000:0.0:1785220582.441846:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+44:2406:x1871938520009728:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.441854:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.441859:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002b8c95ba/0xf62499ab927935f5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fada expref: 44 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441874:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002b8c95ba/0xf62499ab927935f5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fada expref: 44 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441886:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b8c95ba/0xf62499ab927935f5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fada expref: 44 pid: 8441 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441896:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7336:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.441899:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7336:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.441904:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441911:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441930:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.441948:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+43:2406:x1871938520009728:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 102us (789us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.441955:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100803 00010000:00010000:0.0:1785220582.441975:0:14156:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441985:0:14156:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.441994:0:14156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.442002:0:14156:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.442059:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93609500/2, rc 0, transno 4295070834 00000100:00100000:2.0:1785220582.442064:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510801024:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1785220582.442250:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b009500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510803328:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.442282:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.442313:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.442376:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510803840:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.442402:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.442420:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220582.442425:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510803712:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.442445:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.442466:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.442474:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510803584:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.442496:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.442518:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510803328 00000100:00100000:2.0:1785220582.442522:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510803328 00000100:00100000:2.0:1785220582.442546:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510803200 00000100:00100000:2.0:1785220582.442549:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510803200 00000100:00100000:2.0:1785220582.442565:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510803840 00000100:00100000:2.0:1785220582.442568:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510803840 00000100:00100000:2.0:1785220582.442583:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510803712 00000100:00100000:2.0:1785220582.442586:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510803712 00000100:00100000:2.0:1785220582.442601:0:14258:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510803584 00000100:00100000:2.0:1785220582.442604:0:14258:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510803584 00000100:00100000:2.0:1785220582.442621:0:14258:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284872 00000100:00100000:2.0:1785220582.442627:0:14258:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+38:3653:x1871938510803328:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.442647:0:14258:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29497 count 1 00010000:00010000:2.0:1785220582.442665:0:14258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.442680:0:14258:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006eeb28cc/0xf62499ab9279361f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb04 expref: 42 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.442687:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7339:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.442690:0:14258:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7339:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.442694:0:14258:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.442716:0:14258:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006eeb28cc/0xf62499ab9279361f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb04 expref: 42 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.442725:0:14258:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006eeb28cc/0xf62499ab9279361f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb04 expref: 42 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.442736:0:14258:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510803968:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.442786:0:14175:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284873 00000100:00100000:3.0:1785220582.442796:0:14175:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+38:3654:x1871938510803200:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.442809:0:14175:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29496 count 1 00010000:00010000:3.0:1785220582.442830:0:14175:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.442841:0:14175:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dd5778ee/0xf62499ab92793611 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faf6 expref: 42 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.442848:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7338:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.442852:0:14175:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7338:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.442855:0:14175:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.442874:0:14175:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dd5778ee/0xf62499ab92793611 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faf6 expref: 42 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.442884:0:14175:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dd5778ee/0xf62499ab92793611 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faf6 expref: 42 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.442894:0:14175:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d313100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510804096:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.443646:0:14156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.443930:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.443939:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.443951:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.443955:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.443966:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.443987:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.443998:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5c50 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7336:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444024:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.444026:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.444030:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.444055:0:14200:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284874 00000100:00100000:1.0:1785220582.444064:0:14200:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+39:3655:x1871938510803840:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.444099:0:14200:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29500 count 1 00010000:00010000:1.0:1785220582.444116:0:14200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444130:0:14200:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ad3b5146/0xf62499ab92793649 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb2e expref: 42 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444139:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x733c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.444143:0:14200:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.444147:0:14200:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444174:0:14200:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ad3b5146/0xf62499ab92793649 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb2e expref: 42 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444186:0:14200:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ad3b5146/0xf62499ab92793649 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb2e expref: 42 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.444199:0:14200:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d964f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510804224:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.444352:0:14175:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c180 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.444352:0:14258:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.444372:0:14149:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284875 00000100:00100000:2.0:1785220582.444379:0:14149:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+39:3655:x1871938510803712:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.444393:0:14149:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29499 count 1 00010000:00010000:2.0:1785220582.444404:0:14149:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.444442:0:14200:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.444445:0:14149:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b853603b/0xf62499ab9279363b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb20 expref: 42 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.444452:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x733b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.444455:0:14149:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733b:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.444458:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284876 00010000:00010000:2.0:1785220582.444459:0:14149:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.444465:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f726d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+39:3652:x1871938510803584:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.444477:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29498 count 1 00010000:00010000:2.0:1785220582.444478:0:14149:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b853603b/0xf62499ab9279363b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb20 expref: 42 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.444486:0:14149:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b853603b/0xf62499ab9279363b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb20 expref: 42 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444488:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.444497:0:14149:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f727100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510804352:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.444499:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000971dbd6e/0xf62499ab9279362d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb12 expref: 42 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444506:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x733a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.444509:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.444513:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444530:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000971dbd6e/0xf62499ab9279362d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb12 expref: 42 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.444539:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000971dbd6e/0xf62499ab9279362d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb12 expref: 42 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.444548:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d979a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510804480:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.444601:0:14149:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e85436c00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.444601:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.444653:0:14156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+39:3652:x1871938510802432:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17316us (17697us total) trans 4295070263 rc 0/0 00000100:00100000:0.0:1785220582.444661:0:14156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284870 00010000:00010000:0.0:1785220582.444694:0:14186:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.444704:0:14186:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.444741:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d312a00/2, rc 0, transno 4295070263 00000100:00100000:2.0:1785220582.444772:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510802432:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.444836:0:14186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.444844:0:14186:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.444866:0:14186:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.444987:0:14186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_043:lustre-MDT0000-mdtlov_UUID+39:3654:x1871938510802176:12345-0@lo:6:osp-syn-1-0.0 Request processed in 17945us (18108us total) trans 4295070264 rc 0/0 00000100:00100000:0.0:1785220582.444994:0:14186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284868 00000100:00100000:0.0:1785220582.445035:0:14194:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_051:lustre-OST0001_UUID:14194:1871938510803456:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.445056:0:14194:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.445362:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d311f80/2, rc 0, transno 4295070264 00000100:00100000:3.0:1785220582.445372:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d311f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510802176:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.445444:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.445450:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445461:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.445464:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445472:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445487:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445492:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002d8fd624/0xf62499ab927a5c42 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7334:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445512:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.445514:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.445515:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.445526:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510804608:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.445558:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.445584:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.445589:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97985c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510804736:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.445606:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.445719:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.445741:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520009984 00000100:00100000:2.0:1785220582.445756:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520009984 00000100:00100000:3.0:1785220582.445771:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510804608 00000100:00100000:3.0:1785220582.445774:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510804608 00010000:00010000:2.0:1785220582.445781:0:13853:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e7a15f1d/0xf62499ab92793603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae8 expref: 43 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.445806:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100804 00000100:00100000:2.0:1785220582.445814:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9875c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+43:2407:x1871938520009984:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.445825:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.445830:0:13853:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e7a15f1d/0xf62499ab92793603 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae8 expref: 43 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445840:0:13853:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e7a15f1d/0xf62499ab92793603 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae8 expref: 43 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445855:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7a15f1d/0xf62499ab92793603 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fae8 expref: 43 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.445867:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7337:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.445870:0:13853:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7337:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.445875:0:13853:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.445884:0:14173:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510804736 00010000:00010000:2.0:1785220582.445885:0:13853:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.445887:0:14173:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510804736 00010000:00010000:2.0:1785220582.445909:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.445924:0:14206:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284877 00000100:00100000:1.0:1785220582.445931:0:14206:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+40:3652:x1871938510804608:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.445933:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9875c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+42:2407:x1871938520009984:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 121us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.445939:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100804 00002000:00080000:1.0:1785220582.445947:0:14206:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29501 count 1 00010000:00010000:1.0:1785220582.445962:0:14206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.445975:0:14206:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000902dffa7/0xf62499ab92793657 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb3c expref: 41 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.445982:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x733d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.445985:0:14206:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.445988:0:14206:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.446014:0:14206:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000902dffa7/0xf62499ab92793657 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb3c expref: 41 pid: 14176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.446024:0:14206:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000902dffa7/0xf62499ab92793657 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb3c expref: 41 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.446027:0:13818:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284878 00000100:00100000:1.0:1785220582.446035:0:14206:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9875e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510804864:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.446035:0:13818:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+40:3652:x1871938510804736:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.446046:0:13818:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29502 count 1 00010000:00010000:0.0:1785220582.446057:0:13818:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.446066:0:14206:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.446068:0:13818:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000998900bd/0xf62499ab92793665 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb4a expref: 41 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.446075:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x733e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.446078:0:13818:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.446081:0:13818:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.446122:0:14194:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.446132:0:14194:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.446140:0:14194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.446148:0:14194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.446161:0:14194:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1785220582.446255:0:13818:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000998900bd/0xf62499ab92793665 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb4a expref: 41 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.446266:0:13818:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000998900bd/0xf62499ab92793665 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb4a expref: 41 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.446276:0:13818:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510804992:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.446277:0:14194:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_051:lustre-MDT0000-mdtlov_UUID+41:3655:x1871938510803072:12345-0@lo:6:osp-syn-1-0.0 Request processed in 5276us (5405us total) trans 4295070265 rc 0/0 00000100:00100000:1.0:1785220582.446284:0:14194:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284871 00000100:00100000:0.0:1785220582.446302:0:13818:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.446313:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.446318:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.446326:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.446329:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.446338:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.446350:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.446398:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094945f57/0xf62499ab927a5c57 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7337:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.446415:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.446417:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.446418:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.446553:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b008a80/2, rc 0, transno 4295070265 00000100:00100000:2.0:1785220582.446570:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510803072:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.447490:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1785220582.447504:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97986d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510805120:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.447549:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.448652:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510805120 00000100:00100000:1.0:1785220582.448662:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510805120 00000100:00100000:1.0:1785220582.448723:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284879 00000100:00100000:1.0:1785220582.448732:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+41:3655:x1871938510805120:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.448766:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29503 count 1 00010000:00010000:1.0:1785220582.448789:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.448809:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000011f89601/0xf62499ab92793673 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb58 expref: 41 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.448818:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x733f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.448821:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.448825:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.448854:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000011f89601/0xf62499ab92793673 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb58 expref: 41 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.448866:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000011f89601/0xf62499ab92793673 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb58 expref: 41 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.448879:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510805248:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.448949:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896900 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.449543:0:14258:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_081:lustre-OST0001_UUID:14258:1871938510803968:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.449564:0:14258:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.449571:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d979a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510804480:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.449585:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.449588:0:14149:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f727100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:14149:1871938510804352:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.449597:0:14149:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.449647:0:14200:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d964f4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_056:lustre-OST0001_UUID:14200:1871938510804224:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.449660:0:14200:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.449693:0:14175:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d313100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_036:lustre-OST0001_UUID:14175:1871938510804096:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.449710:0:14175:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.449770:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.449802:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.449838:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.449865:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.449881:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.449917:0:14206:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9875e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_062:lustre-OST0001_UUID:14206:1871938510804864:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.449923:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520010112 00000100:00100000:0.0:1785220582.449927:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520010112 00010000:00010000:1.0:1785220582.449930:0:14206:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.449950:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006eeb28cc/0xf62499ab9279361f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb04 expref: 42 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.449973:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.449977:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520010368 00000100:00100000:0.0:1785220582.449980:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520010368 00010000:00010000:0.0:1785220582.449991:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ad3b5146/0xf62499ab92793649 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb2e expref: 43 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.450018:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520010624 00000100:00100000:0.0:1785220582.450022:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520010624 00010000:00010000:0.0:1785220582.450034:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000971dbd6e/0xf62499ab9279362d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb12 expref: 44 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.450051:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520010240 00000100:00100000:0.0:1785220582.450054:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520010240 00000100:00100000:3.0:1785220582.450064:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520010496 00010000:00010000:0.0:1785220582.450065:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dd5778ee/0xf62499ab92793611 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faf6 expref: 45 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.450068:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520010496 00000100:00100000:0.0:1785220582.450080:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520010752 00010000:00010000:3.0:1785220582.450082:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b853603b/0xf62499ab9279363b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb20 expref: 46 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.450082:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520010752 00010000:00010000:0.0:1785220582.450127:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000902dffa7/0xf62499ab92793657 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb3c expref: 47 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.450142:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100805 00000100:00100000:0.0:1785220582.450146:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100806 00000100:00100000:3.0:1785220582.450150:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2406:x1871938520010112:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.450152:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e10450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+47:2406:x1871938520010368:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.450162:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.450162:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.450168:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006eeb28cc/0xf62499ab9279361f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb04 expref: 47 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450168:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ad3b5146/0xf62499ab92793649 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb2e expref: 47 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.450179:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006eeb28cc/0xf62499ab9279361f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb04 expref: 47 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450183:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ad3b5146/0xf62499ab92793649 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb2e expref: 47 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.450202:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006eeb28cc/0xf62499ab9279361f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb04 expref: 47 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450202:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad3b5146/0xf62499ab92793649 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb2e expref: 47 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.450217:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7339:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.450217:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.450221:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7339:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.450221:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.450227:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450227:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.450237:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450237:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.450254:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.450256:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.450288:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e10450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+45:2406:x1871938520010368:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 134us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.450295:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100806 00000100:00100000:1.0:1785220582.450310:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100807 00000100:00100000:1.0:1785220582.450317:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+44:2406:x1871938520010624:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.450325:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.450331:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000971dbd6e/0xf62499ab9279362d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb12 expref: 44 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.450340:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000971dbd6e/0xf62499ab9279362d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb12 expref: 44 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.450354:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000971dbd6e/0xf62499ab9279362d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb12 expref: 44 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.450363:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.450367:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.450370:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.450378:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.450393:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.450762:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100808 00000100:00100000:0.0:1785220582.450772:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+43:2406:x1871938520010240:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.450780:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.450786:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dd5778ee/0xf62499ab92793611 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faf6 expref: 43 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450799:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dd5778ee/0xf62499ab92793611 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faf6 expref: 43 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450812:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dd5778ee/0xf62499ab92793611 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7faf6 expref: 43 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450822:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7338:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.450824:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7338:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.450828:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450835:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.450853:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.450884:0:14211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100809 00000100:00100000:2.0:1785220582.450893:0:14211:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+42:2406:x1871938520010496:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.450901:0:14211:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.450906:0:14211:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b853603b/0xf62499ab9279363b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb20 expref: 42 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.450921:0:14211:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b853603b/0xf62499ab9279363b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb20 expref: 42 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.450935:0:14211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b853603b/0xf62499ab9279363b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb20 expref: 42 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.450944:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.450948:0:14211:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.450952:0:14211:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.450960:0:14211:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.450977:0:14211:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.451574:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+41:2406:x1871938520010112:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1422us (1801us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.451582:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100805 00000100:00100000:3.0:1785220582.451713:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100810 00000100:00100000:3.0:1785220582.451724:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+40:2406:x1871938520010752:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.451734:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.451743:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000902dffa7/0xf62499ab92793657 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb3c expref: 40 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.451766:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000902dffa7/0xf62499ab92793657 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb3c expref: 40 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.451792:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000902dffa7/0xf62499ab92793657 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb3c expref: 40 pid: 14176 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.451933:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.451962:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.451992:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.452002:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.452004:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+39:2406:x1871938520010240:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1234us (2138us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.452012:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100808 00010000:00010000:3.0:1785220582.452023:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.452044:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+38:2406:x1871938520010752:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 321us (2070us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.452049:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100810 00000100:00100000:0.0:1785220582.452057:0:13818:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13818:1871938510804992:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.452074:0:14175:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.452075:0:13818:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.452170:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+37:2406:x1871938520010624:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1853us (2330us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.452178:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100807 00010000:00010000:1.0:1785220582.452199:0:14206:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.452206:0:14200:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452207:0:14206:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452218:0:14206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.452220:0:14200:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452227:0:14206:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.452228:0:14200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.452235:0:14200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452243:0:14206:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1785220582.452246:0:14200:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220582.452354:0:14175:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.452365:0:14175:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.452372:0:14175:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.452387:0:14175:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1785220582.452427:0:14211:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_010:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2406:x1871938520010496:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1534us (2543us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.452434:0:14211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100809 00010000:00010000:2.0:1785220582.452462:0:14149:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452471:0:14149:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452478:0:14149:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452485:0:14149:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.452494:0:14175:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d312300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_036:lustre-MDT0000-mdtlov_UUID+44:3654:x1871938510803200:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9701us (10180us total) trans 4295070268 rc 0/0 00010000:00010000:2.0:1785220582.452497:0:14149:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:3.0:1785220582.452502:0:14175:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284873 00000100:00100000:2.0:1785220582.452583:0:14149:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0000-mdtlov_UUID+44:3655:x1871938510803712:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8203us (10136us total) trans 4295070269 rc 0/0 00000100:00100000:2.0:1785220582.452589:0:14149:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284875 00000100:00100000:1.0:1785220582.452592:0:14206:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_062:lustre-MDT0000-mdtlov_UUID+44:3652:x1871938510804608:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6662us (7034us total) trans 4295070267 rc 0/0 00000100:00100000:1.0:1785220582.452597:0:14206:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284877 00010000:00010000:2.0:1785220582.452608:0:14258:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452617:0:14258:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452625:0:14258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452632:0:14258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452641:0:14258:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 00010000:00010000:1.0:1785220582.452684:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1785220582.452690:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452699:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.452703:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.452713:0:14258:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_081:lustre-MDT0000-mdtlov_UUID+43:3653:x1871938510803328:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10086us (10430us total) trans 4295070270 rc 0/0 00010000:00010000:1.0:1785220582.452713:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.452720:0:14258:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284872 00010000:00010000:1.0:1785220582.452727:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452734:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5c81 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14206 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452736:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1785220582.452740:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452759:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.452762:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452764:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.452768:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452771:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452775:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.452778:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452783:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452785:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452791:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005775584e/0xf62499ab927a5c6c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452801:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452806:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.452809:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.452809:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c2168cb/0xf62499ab927a5c65 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7338:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452816:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.452819:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452827:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452835:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452842:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5c73 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14149 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452854:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.452857:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452864:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.452867:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452874:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452884:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452891:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5c5e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7339:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.452903:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.452905:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.452906:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.452917:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d93609f80/2, rc 0, transno 4295070269 00000004:00080000:2.0:1785220582.452922:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8d312680/2, rc 0, transno 4295070267 00000100:00100000:3.0:1785220582.452927:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93609f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510803712:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.452927:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510804608:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.452965:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b009500/2, rc 0, transno 4295070270 00000100:00100000:2.0:1785220582.452968:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b009500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510803328:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.452996:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97984380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510805504:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.453007:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97987100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510805376:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.453027:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.453053:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.453055:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.453056:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.453060:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:1.0:1785220582.453062:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.453066:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97985500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510805632:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.453073:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453081:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.453236:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1785220582.453252:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453260:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453276:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.453281:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.453285:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.453292:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.453295:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.453303:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.453314:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.453321:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b5b90b3c/0xf62499ab927a5c7a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.453335:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.453337:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.453339:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.453353:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f726d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+43:3652:x1871938510803584:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8888us (10856us total) trans 4295070271 rc 0/0 00000004:00080000:2.0:1785220582.453359:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b008000/2, rc 0, transno 4295070271 00000100:00100000:1.0:1785220582.453363:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284876 00000100:00100000:2.0:1785220582.453367:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b008000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510803584:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.453377:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510805504 00000100:00100000:1.0:1785220582.453383:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510805504 00000100:00100000:1.0:1785220582.453420:0:14165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510805632 00000100:00100000:1.0:1785220582.453423:0:14165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510805632 00000100:00100000:2.0:1785220582.453434:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284880 00000100:00100000:1.0:1785220582.453441:0:14165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284881 00000100:00100000:2.0:1785220582.453442:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+44:3652:x1871938510805504:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.453446:0:14165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f727800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+44:3652:x1871938510805632:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.453462:0:14165:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29506 count 1 00010000:00010000:1.0:1785220582.453480:0:14165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453495:0:14165:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e4d6fb39/0xf62499ab9279369d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb82 expref: 35 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453502:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7342:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.453506:0:14165:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7342:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.453510:0:14165:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453533:0:14165:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e4d6fb39/0xf62499ab9279369d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb82 expref: 35 pid: 14191 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.453538:0:14200:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_056:lustre-MDT0000-mdtlov_UUID+44:3655:x1871938510803840:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9476us (11134us total) trans 4295070266 rc 0/0 00000100:00100000:0.0:1785220582.453545:0:14200:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284874 00010000:00010000:1.0:1785220582.453546:0:14165:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e4d6fb39/0xf62499ab9279369d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb82 expref: 35 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.453557:0:14165:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510805888:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.453566:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.453596:0:14165:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.453660:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.453679:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011008 00000100:00100000:0.0:1785220582.453683:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011008 00000100:00100000:3.0:1785220582.453690:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.453701:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000011f89601/0xf62499ab92793673 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb58 expref: 36 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220582.453716:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9360b480/2, rc 0, transno 4295070266 00000100:00100000:3.0:1785220582.453722:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510803840:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.453723:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100811 00000100:00100000:3.0:1785220582.453729:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.453729:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2406:x1871938520011008:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1785220582.453734:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510805760:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.453737:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.453743:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000011f89601/0xf62499ab92793673 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb58 expref: 36 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.453767:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.453769:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000011f89601/0xf62499ab92793673 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb58 expref: 36 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.453783:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.453787:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.453792:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.453800:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.453807:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510805376 00000100:00100000:1.0:1785220582.453811:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510805376 00010000:00010000:0.0:1785220582.453815:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.453833:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2406:x1871938520011008:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (174us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.453839:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100811 00000100:00100000:0.0:1785220582.453884:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.453899:0:14188:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510805760 00000100:00100000:1.0:1785220582.453903:0:14188:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510805760 00000100:00100000:1.0:1785220582.453921:0:14188:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284882 00000100:00100000:1.0:1785220582.453927:0:14188:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97987800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+45:3655:x1871938510805376:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.453939:0:14188:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29504 count 1 00010000:00010000:1.0:1785220582.453949:0:14188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453960:0:14188:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000db28c4be/0xf62499ab92793681 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb66 expref: 35 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453966:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7340:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.453969:0:14188:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7340:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.453972:0:14188:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.453990:0:14188:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000db28c4be/0xf62499ab92793681 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb66 expref: 35 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.454000:0:14188:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000db28c4be/0xf62499ab92793681 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb66 expref: 35 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220582.454009:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9b00b480/2, rc 0, transno 4295070268 00000100:00100000:1.0:1785220582.454010:0:14188:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510806016:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.454013:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9b00b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510803200:0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.454074:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29505 count 1 00000100:00100000:3.0:1785220582.454363:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520010880 00000100:00100000:3.0:1785220582.454368:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520010880 00010000:00010000:3.0:1785220582.454385:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000998900bd/0xf62499ab92793665 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb4a expref: 36 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.454407:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100812 00000100:00100000:3.0:1785220582.454414:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2404:x1871938520010880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.454422:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.454426:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000998900bd/0xf62499ab92793665 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb4a expref: 36 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.454442:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000998900bd/0xf62499ab92793665 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb4a expref: 36 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.454456:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000998900bd/0xf62499ab92793665 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb4a expref: 36 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.454467:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.454471:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x733e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.454475:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.454483:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.454484:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.454495:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f9eb4329/0xf62499ab9279368f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb74 expref: 35 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.454502:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220582.454502:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7341:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.454505:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7341:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.454508:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.454526:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f9eb4329/0xf62499ab9279368f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb74 expref: 35 pid: 14175 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.454535:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f9eb4329/0xf62499ab9279368f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb74 expref: 35 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.454545:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510806144:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.454884:0:14188:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.454922:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000011f89601/0xf62499ab92793673 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb58 expref: 35 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.454934:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510805248:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.454954:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.454967:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.454974:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.454982:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.454992:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.455006:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.455153:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+46:3655:x1871938510805120:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6421us (7602us total) trans 4295070272 rc 0/0 00000100:00100000:1.0:1785220582.455161:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284879 00000100:00100000:1.0:1785220582.455180:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284883 00000100:00100000:1.0:1785220582.455187:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97986a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+45:3655:x1871938510805760:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.455202:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29507 count 1 00010000:00010000:1.0:1785220582.455216:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.455228:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f487e047/0xf62499ab927936ab lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb90 expref: 34 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.455234:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7343:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.455237:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7343:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.455241:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.455261:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f487e047/0xf62499ab927936ab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb90 expref: 34 pid: 14223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.455271:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f487e047/0xf62499ab927936ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb90 expref: 34 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.455282:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510806272:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.456028:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+34:2404:x1871938520010880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1611us (2140us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.456037:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100812 00010000:00010000:3.0:1785220582.456073:0:13818:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.456106:0:13818:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.456117:0:13818:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.456124:0:13818:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.456145:0:13818:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1785220582.456272:0:13818:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+46:3652:x1871938510804736:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10237us (10663us total) trans 4295070273 rc 0/0 00000100:00100000:3.0:1785220582.456280:0:13818:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284878 00000004:00080000:3.0:1785220582.456323:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97986d80/2, rc 0, transno 4295070272 00000100:00100000:3.0:1785220582.456329:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97986d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510805120:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.456970:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e85436c00 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.457083:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa62300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510806528:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.457134:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa61f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510806400:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.457177:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.457277:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.457310:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97985c00/2, rc 0, transno 4295070273 00000100:00100000:2.0:1785220582.457325:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97985c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510804736:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.457333:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.457338:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510806656:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.457355:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.457431:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896540 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.457437:0:14179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510806400 00000100:00100000:0.0:1785220582.457443:0:14179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510806400 00000100:00100000:1.0:1785220582.457445:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510806528 00000100:00100000:1.0:1785220582.457447:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510806528 00000100:00100000:1.0:1785220582.457533:0:13908:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510806656 00000100:00100000:1.0:1785220582.457537:0:13908:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510806656 00000100:00100000:0.0:1785220582.457539:0:14179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284884 00000100:00100000:0.0:1785220582.457546:0:14179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+48:3652:x1871938510806528:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.457559:0:13908:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284885 00000100:00100000:1.0:1785220582.457564:0:13908:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8ecea850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+48:3655:x1871938510806400:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.457568:0:14179:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29509 count 1 00002000:00080000:1.0:1785220582.457575:0:13908:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29508 count 1 00010000:00010000:1.0:1785220582.457588:0:13908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457588:0:14179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457606:0:14179:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003836d788/0xf62499ab927936c7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbac expref: 33 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457614:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7345:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.457619:0:14179:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7345:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.457623:0:14179:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457649:0:14179:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003836d788/0xf62499ab927936c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbac expref: 33 pid: 13874 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457661:0:14179:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003836d788/0xf62499ab927936c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbac expref: 33 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.457672:0:14179:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510806784:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.457702:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1785220582.457708:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97984000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510806912:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.457713:0:14179:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f000 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.457728:0:14190:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284886 00000100:00100000:3.0:1785220582.457730:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.457736:0:14190:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+48:3652:x1871938510806656:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.457760:0:14190:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29510 count 1 00000100:00100000:3.0:1785220582.457761:0:13909:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510806912 00000100:00100000:3.0:1785220582.457765:0:13909:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510806912 00010000:00010000:0.0:1785220582.457772:0:14190:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457783:0:14190:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000be83f143/0xf62499ab927936d5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbba expref: 33 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.457786:0:13909:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284887 00000100:00100000:3.0:1785220582.457792:0:13909:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97986680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+49:3655:x1871938510806912:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.457797:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7346:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.457800:0:14190:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7346:0x0].0x0 since=0s v=(1 0) 00002000:00080000:3.0:1785220582.457802:0:13909:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29511 count 1 00010000:00010000:0.0:1785220582.457804:0:14190:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.457813:0:13909:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457821:0:14190:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000be83f143/0xf62499ab927936d5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbba expref: 33 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.457830:0:14190:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000be83f143/0xf62499ab927936d5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbba expref: 33 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.457835:0:13909:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000089f8a195/0xf62499ab927936e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc8 expref: 33 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.457840:0:14190:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a706680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510807040:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.457848:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7347:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.457850:0:13909:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7347:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.457854:0:13909:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.457861:0:14190:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00010000:00010000:3.0:1785220582.457871:0:13909:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000089f8a195/0xf62499ab927936e3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc8 expref: 33 pid: 14258 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.457879:0:13909:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000089f8a195/0xf62499ab927936e3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc8 expref: 33 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.457888:0:13909:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97984a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510807168:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.457916:0:13908:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000038f6aea5/0xf62499ab927936b9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb9e expref: 33 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.457923:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7344:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.457927:0:13908:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7344:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.457931:0:13908:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.457947:0:13908:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000038f6aea5/0xf62499ab927936b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb9e expref: 33 pid: 14150 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.457956:0:13908:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000038f6aea5/0xf62499ab927936b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb9e expref: 33 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.457964:0:13908:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510807296:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.458062:0:13909:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c300 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.458204:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.458212:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458221:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.458225:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458235:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458253:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458261:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000927db29/0xf62499ab927a5c8f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458280:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.458283:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458293:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.458296:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458305:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458315:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458322:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004765897f/0xf62499ab927a5c88 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x733e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.458335:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.458337:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.458339:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.458345:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.458413:0:13908:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8969c0 going to sleep for 11 seconds 00000100:00100000:1.0:1785220582.463413:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.463465:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.463499:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.463525:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011136 00000100:00100000:1.0:1785220582.463534:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011136 00010000:00010000:1.0:1785220582.463573:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e4d6fb39/0xf62499ab9279369d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb82 expref: 34 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.463616:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011264 00000100:00100000:1.0:1785220582.463620:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011264 00010000:00010000:1.0:1785220582.463632:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000db28c4be/0xf62499ab92793681 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb66 expref: 35 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.463650:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011392 00000100:00100000:1.0:1785220582.463652:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011392 00000100:00100000:0.0:1785220582.463654:0:14188:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_045:lustre-OST0001_UUID:14188:1871938510806016:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.463662:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f9eb4329/0xf62499ab9279368f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb74 expref: 36 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.463676:0:14188:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.463680:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100813 00000100:00100000:1.0:1785220582.463688:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e90703450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2407:x1871938520011136:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.463706:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.463712:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e4d6fb39/0xf62499ab9279369d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb82 expref: 36 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463721:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e4d6fb39/0xf62499ab9279369d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb82 expref: 36 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463743:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7342:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.463760:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7342:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.463766:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463779:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463798:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.463810:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f725f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510806144:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.463835:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.463852:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e90703450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+36:2407:x1871938520011136:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 164us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.463859:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100813 00010000:00010000:2.0:1785220582.463863:0:14165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e4d6fb39/0xf62499ab9279369d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb82 expref: 36 pid: 14191 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.463878:0:14165:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_027:lustre-OST0001_UUID:14165:1871938510805888:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.463887:0:14165:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.463895:0:14165:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.463902:0:14165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.463903:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100814 00000100:00100000:1.0:1785220582.463909:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+34:2406:x1871938520011264:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.463912:0:14165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463916:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220582.463921:0:14165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463921:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000db28c4be/0xf62499ab92793681 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb66 expref: 34 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463931:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000db28c4be/0xf62499ab92793681 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb66 expref: 34 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.463937:0:14165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.463942:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db28c4be/0xf62499ab92793681 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb66 expref: 34 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463953:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7340:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.463955:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7340:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.463959:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463966:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.463983:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.464000:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+33:2406:x1871938520011264:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 91us (535us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.464006:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100814 00000100:00100000:1.0:1785220582.464017:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100815 00000100:00100000:1.0:1785220582.464023:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+32:2407:x1871938520011392:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.464030:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.464034:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f9eb4329/0xf62499ab9279368f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb74 expref: 32 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.464043:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f9eb4329/0xf62499ab9279368f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb74 expref: 32 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.464055:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f9eb4329/0xf62499ab9279368f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb74 expref: 32 pid: 14175 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.464064:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7341:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.464067:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7341:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.464071:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.464078:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.464237:0:14188:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.464246:0:14188:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.464258:0:14188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.464263:0:14165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_027:lustre-MDT0000-mdtlov_UUID+50:3652:x1871938510805632:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10815us (11176us total) trans 4295070274 rc 0/0 00010000:00010000:0.0:1785220582.464266:0:14188:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.464271:0:14165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284881 00010000:00010000:0.0:1785220582.464279:0:14188:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1785220582.464291:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.464297:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464305:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.464309:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464324:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464336:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464343:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008a72457b/0xf62499ab927a5c96 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7342:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.464355:0:14188:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97987800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_045:lustre-MDT0000-mdtlov_UUID+50:3655:x1871938510805376:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10429us (10665us total) trans 4295070275 rc 0/0 00010000:00010000:2.0:1785220582.464361:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1785220582.464362:0:14188:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284882 00010000:00010000:2.0:1785220582.464364:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464370:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.464373:0:10345:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.464377:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.464381:0:10345:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464390:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464398:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a048043/0xf62499ab927a5c9d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7340:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14188 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.464419:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.464420:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.464422:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.464453:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97985500/2, rc 0, transno 4295070274 00000100:00100000:2.0:1785220582.464468:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97985500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510805632:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.464488:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.464586:0:14179:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_040:lustre-OST0001_UUID:14179:1871938510806784:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.464600:0:14179:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.464637:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011520 00000100:00100000:3.0:1785220582.464642:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011520 00000100:00100000:0.0:1785220582.464656:0:14190:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a706680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_047:lustre-OST0001_UUID:14190:1871938510807040:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.464662:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f487e047/0xf62499ab927936ab lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb90 expref: 32 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.464667:0:14190:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.464709:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.465018:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.465045:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+32:2407:x1871938520011392:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1021us (1545us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.465052:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100815 00000100:00100000:3.0:1785220582.465709:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011648 00000100:00100000:3.0:1785220582.465715:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011648 00010000:00010000:3.0:1785220582.465739:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003836d788/0xf62499ab927936c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbac expref: 32 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.465780:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100816 00000100:00100000:3.0:1785220582.465788:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+32:2406:x1871938520011520:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.465803:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.465808:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f487e047/0xf62499ab927936ab lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb90 expref: 32 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.465821:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f487e047/0xf62499ab927936ab lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb90 expref: 32 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.465840:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7343:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.465844:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7343:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.465849:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.465858:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.465872:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.465901:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+32:2406:x1871938520011520:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 113us (1412us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.465910:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100816 00000100:00100000:3.0:1785220582.465930:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100817 00000100:00100000:3.0:1785220582.465936:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+31:2407:x1871938520011648:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.465944:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.465951:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003836d788/0xf62499ab927936c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbac expref: 31 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.465969:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003836d788/0xf62499ab927936c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbac expref: 31 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.465984:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003836d788/0xf62499ab927936c7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbac expref: 31 pid: 13874 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.465998:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7345:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.466001:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7345:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.466007:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466015:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466035:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.466055:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+30:2407:x1871938520011648:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 119us (1346us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.466064:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100817 00010000:00010000:3.0:1785220582.466199:0:14179:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466213:0:14179:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466222:0:14179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466231:0:14179:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466251:0:14179:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.466416:0:14179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_040:lustre-MDT0000-mdtlov_UUID+50:3652:x1871938510806528:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8869us (9138us total) trans 4295070276 rc 0/0 00000100:00100000:3.0:1785220582.466427:0:14179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284884 00010000:00010000:3.0:1785220582.466449:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.466454:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466465:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.466470:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466478:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466492:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466501:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000005ee637/0xf62499ab927a5cb2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7345:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.466521:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.466523:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.466525:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.466555:0:13909:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97984a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:13909:1871938510807168:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.466575:0:13909:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.466613:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa63b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510807424:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.466646:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220582.466688:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97987100/2, rc 0, transno 4295070275 00000100:00100000:3.0:1785220582.466698:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510805376:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.466792:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.466802:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.466809:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f487e047/0xf62499ab927936ab lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb90 expref: 29 pid: 14223 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.466810:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.466818:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.466876:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.466892:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510806272:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.466908:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.466917:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.466923:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.466931:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.466938:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.466959:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f727100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+50:3652:x1871938510805504:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13517us (13931us total) trans 4295070277 rc 0/0 00000100:00100000:2.0:1785220582.466966:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284880 00000100:00100000:2.0:1785220582.466978:0:13866:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510807424 00000100:00100000:2.0:1785220582.466983:0:13866:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510807424 00000100:00100000:2.0:1785220582.467007:0:13866:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284888 00000100:00100000:2.0:1785220582.467014:0:13866:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97984a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+50:3655:x1871938510807424:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.467014:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.467031:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011904 00000100:00100000:0.0:1785220582.467034:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011904 00002000:00080000:2.0:1785220582.467036:0:13866:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29512 count 1 00010000:00010000:0.0:1785220582.467051:0:14239:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000089f8a195/0xf62499ab927936e3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc8 expref: 29 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.467052:0:13866:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.467066:0:13866:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000170925e5/0xf62499ab927936f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbd6 expref: 29 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.467072:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100818 00010000:00010000:2.0:1785220582.467073:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7348:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.467077:0:13866:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7348:0x0].0x0 since=0s v=(1 0) 00000100:00100000:0.0:1785220582.467078:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+29:2407:x1871938520011904:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.467081:0:13866:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.467200:0:13866:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000170925e5/0xf62499ab927936f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbd6 expref: 29 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.467213:0:13866:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000170925e5/0xf62499ab927936f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbd6 expref: 29 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467220:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1785220582.467225:0:13866:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa61180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510807552:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.467226:0:14239:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000089f8a195/0xf62499ab927936e3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc8 expref: 29 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467236:0:14239:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000089f8a195/0xf62499ab927936e3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc8 expref: 29 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467249:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000089f8a195/0xf62499ab927936e3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbc8 expref: 29 pid: 14258 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467258:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7347:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.467261:0:14239:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7347:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1785220582.467262:0:13866:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a4d5600 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.467265:0:14239:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467273:0:14239:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467292:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.467311:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+28:2407:x1871938520011904:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 233us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.467313:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510807680:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.467321:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100818 00010000:00010000:0.0:1785220582.467338:0:13909:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.467339:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220582.467345:0:13909:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467353:0:13909:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.467360:0:13909:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.467363:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa62300/2, rc 0, transno 4295070276 00000100:00100000:2.0:1785220582.467369:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa62300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510806528:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.467405:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97984380/2, rc 0, transno 4295070277 00000100:00100000:2.0:1785220582.467413:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97984380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510805504:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.467447:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1785220582.467453:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510807808:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.467482:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220582.467640:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1785220582.467681:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467690:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.467691:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1785220582.467694:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467701:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467713:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467720:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000492e9ee/0xf62499ab927a5ca4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7341:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467741:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1785220582.467755:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467762:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.467765:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467772:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467781:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467788:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000044a9f3cf/0xf62499ab927a5cab lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7343:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.467790:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97986a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+51:3655:x1871938510805760:12345-0@lo:6:osp-syn-1-0.0 Request processed in 12604us (14022us total) trans 4295070278 rc 0/0 00000100:00100000:1.0:1785220582.467798:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284883 00010000:00010000:3.0:1785220582.467802:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220582.467805:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.467810:0:14249:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510807680 00010000:00010000:3.0:1785220582.467811:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68000 already unused, calling callback (ffffffffc0e5c310) 00000100:00100000:1.0:1785220582.467813:0:14249:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510807680 00010000:00010000:3.0:1785220582.467814:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467827:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467836:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467844:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.467846:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.467848:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.467858:0:14177:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510807808 00000100:00100000:3.0:1785220582.467862:0:14177:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510807808 00010000:00010000:2.0:1785220582.467883:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.467904:0:14249:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284889 00010000:00010000:0.0:1785220582.467933:0:13909:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1785220582.467938:0:13909:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c028e57/0xf62499ab927a5cc7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7347:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.467942:0:14177:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284890 00000100:00100000:3.0:1785220582.467948:0:14177:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+52:3652:x1871938510807808:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1785220582.467963:0:14177:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29514 count 1 00010000:00010000:3.0:1785220582.467976:0:14177:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467987:0:14177:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000064c7d507/0xf62499ab9279370d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf2 expref: 27 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.467995:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x734a:0x0].0x0 since=17s v=(0 0) 00010000:00010000:3.0:1785220582.467998:0:14177:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.468002:0:14177:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468021:0:14177:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000064c7d507/0xf62499ab9279370d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf2 expref: 27 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.468021:0:13909:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97986680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+53:3655:x1871938510806912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10229us (10289us total) trans 4295070279 rc 0/0 00000100:00100000:0.0:1785220582.468028:0:13909:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284887 00010000:00010000:3.0:1785220582.468031:0:14177:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000064c7d507/0xf62499ab9279370d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf2 expref: 27 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.468041:0:14177:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510808064:192.168.201.51@tcp:104: 00000100:00100000:3.0:1785220582.468066:0:14177:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38c300 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.468069:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.468120:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:2.0:1785220582.468248:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.468285:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97984000/2, rc 0, transno 4295070279 00000100:00100000:3.0:1785220582.468326:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97984000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510806912:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.468338:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520011776 00000100:00100000:3.0:1785220582.468342:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520011776 00010000:00010000:3.0:1785220582.468360:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000be83f143/0xf62499ab927936d5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbba expref: 28 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.468384:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012032 00000100:00100000:3.0:1785220582.468387:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012032 00010000:00010000:3.0:1785220582.468456:0:13923:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000038f6aea5/0xf62499ab927936b9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb9e expref: 29 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.468487:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100819 00000100:00100000:3.0:1785220582.468495:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+29:2407:x1871938520011776:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.468508:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.468514:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000be83f143/0xf62499ab927936d5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbba expref: 29 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468525:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000be83f143/0xf62499ab927936d5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbba expref: 29 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468540:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000be83f143/0xf62499ab927936d5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbba expref: 29 pid: 14189 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468551:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7346:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.468554:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7346:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.468558:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468566:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468582:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.468601:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+28:2407:x1871938520011776:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 109us (532us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.468607:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100819 00000100:00100000:3.0:1785220582.468621:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100820 00000100:00100000:3.0:1785220582.468628:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+27:2406:x1871938520012032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.468634:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.468638:0:13923:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000038f6aea5/0xf62499ab927936b9 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb9e expref: 27 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468647:0:13923:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000038f6aea5/0xf62499ab927936b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb9e expref: 27 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468662:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7344:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.468665:0:13923:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7344:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.468669:0:13923:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468677:0:13923:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.468687:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.468704:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+27:2406:x1871938520012032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 77us (585us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.468711:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100820 00000004:00080000:3.0:1785220582.468769:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8f724000/2, rc 0, transno 4295070278 00010000:00010000:0.0:1785220582.468788:0:14190:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.468798:0:14190:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.468807:0:14190:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.468813:0:14190:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.468840:0:14190:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.468927:0:14190:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_047:lustre-MDT0000-mdtlov_UUID+53:3652:x1871938510806656:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11192us (11570us total) trans 4295070280 rc 0/0 00000100:00100000:0.0:1785220582.468934:0:14190:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284886 00000100:00100000:1.0:1785220582.469762:0:14249:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+52:3652:x1871938510807680:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.469781:0:14249:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29513 count 1 00000100:00100000:2.0:1785220582.469784:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510808320:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.469799:0:14249:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.469813:0:14249:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cd1b4aac/0xf62499ab927936ff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbe4 expref: 26 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.469815:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1785220582.469854:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa60000/2, rc 0, transno 4295070280 00000100:00100000:2.0:1785220582.469863:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510806656:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.469891:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220582.469895:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510808320 00000100:00100000:2.0:1785220582.469897:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa60e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510808448:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.469899:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510808320 00000100:00100000:2.0:1785220582.469920:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.469926:0:13837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510808448 00000100:00100000:0.0:1785220582.469929:0:13837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510808448 00000100:00100000:0.0:1785220582.469945:0:13837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284891 00000100:00100000:0.0:1785220582.469952:0:13837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+54:3652:x1871938510808320:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.469964:0:13837:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29517 count 1 00010000:00010000:0.0:1785220582.469976:0:13837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.469987:0:13837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000740c1176/0xf62499ab92793737 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc1c expref: 26 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.469994:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x734d:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.469997:0:13837:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.470001:0:13837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470024:0:13837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000740c1176/0xf62499ab92793737 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc1c expref: 26 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470041:0:13837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000740c1176/0xf62499ab92793737 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc1c expref: 26 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.470053:0:13837:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510808576:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.470180:0:13837:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.470197:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284892 00000100:00100000:0.0:1785220582.470205:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+54:3652:x1871938510808448:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.470216:0:8752:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29518 count 1 00010000:00010000:0.0:1785220582.470228:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470240:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000076ced996/0xf62499ab92793745 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc2a expref: 26 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470247:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x734e:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.470250:0:8752:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.470253:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470273:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000076ced996/0xf62499ab92793745 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc2a expref: 26 pid: 14261 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.470274:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.470280:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470282:0:8752:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000076ced996/0xf62499ab92793745 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc2a expref: 26 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.470289:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.470293:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.470293:0:8752:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510808704:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.470303:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.470318:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.470318:0:8752:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fcc0 going to sleep for 11 seconds 00010000:00010000:2.0:1785220582.470331:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2ad84e0/0xf62499ab927a5cc0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7346:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.470349:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.470351:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.470353:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.470473:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8f724000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510805760:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.470583:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa61500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510808192:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.470612:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220582.470641:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1785220582.470647:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8fa63480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510807936:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.470668:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.470691:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510808192 00000100:00100000:0.0:1785220582.470695:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510808192 00000100:00100000:0.0:1785220582.470719:0:14162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510807936 00000100:00100000:0.0:1785220582.470722:0:14162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510807936 00000100:00100000:0.0:1785220582.470758:0:14162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284893 00000100:00100000:0.0:1785220582.470764:0:14162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97985180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+56:3655:x1871938510808192:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.470776:0:14162:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29516 count 1 00010000:00010000:0.0:1785220582.470788:0:14162:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470800:0:14162:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000348e48b0/0xf62499ab92793729 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc0e expref: 26 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470807:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x734c:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.470810:0:14162:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.470814:0:14162:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470844:0:14162:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000348e48b0/0xf62499ab92793729 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc0e expref: 26 pid: 13909 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470857:0:14162:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000348e48b0/0xf62499ab92793729 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc0e expref: 26 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.470866:0:14162:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510808832:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.470903:0:14162:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fc00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.470920:0:14266:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284894 00000100:00100000:0.0:1785220582.470928:0:14266:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97986300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+56:3655:x1871938510807936:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.470941:0:14266:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29515 count 1 00010000:00010000:0.0:1785220582.470952:0:14266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470962:0:14266:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001ff5b905/0xf62499ab9279371b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc00 expref: 26 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470968:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x734b:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.470971:0:14266:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.470974:0:14266:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470989:0:14266:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001ff5b905/0xf62499ab9279371b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc00 expref: 26 pid: 14200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.470997:0:14266:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001ff5b905/0xf62499ab9279371b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc00 expref: 26 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.471011:0:14266:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a706300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510808960:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.471032:0:14266:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f9c0 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.471234:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7349:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.471237:0:14249:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7349:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.471241:0:14249:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.471262:0:14249:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cd1b4aac/0xf62499ab927936ff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbe4 expref: 26 pid: 14248 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.471272:0:14249:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cd1b4aac/0xf62499ab927936ff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbe4 expref: 26 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.471283:0:14249:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510809088:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.471308:0:14249:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896900 going to sleep for 11 seconds 00010000:00010000:1.0:1785220582.471337:0:13908:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038f6aea5/0xf62499ab927936b9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fb9e expref: 26 pid: 14150 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.471351:0:13908:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:13908:1871938510807296:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.471369:0:13908:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.471380:0:13908:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.471387:0:13908:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.471396:0:13908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.471404:0:13908:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.471428:0:13908:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220582.471508:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.471514:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471521:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.471524:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471531:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471543:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471550:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000068b232dd/0xf62499ab927a5cb9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7344:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471566:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.471568:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.471570:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.471583:0:13908:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8ecea850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+57:3655:x1871938510806400:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14017us (14405us total) trans 4295070281 rc 0/0 00000100:00100000:1.0:1785220582.471591:0:13908:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284885 00000004:00080000:3.0:1785220582.471613:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa61f80/2, rc 0, transno 4295070281 00000100:00100000:3.0:1785220582.471627:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa61f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510806400:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.471727:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1785220582.471736:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809216:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1785220582.471772:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.471831:0:14172:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510809216 00000100:00100000:2.0:1785220582.471836:0:14172:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510809216 00000100:00100000:2.0:1785220582.471872:0:14172:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284895 00000100:00100000:2.0:1785220582.471880:0:14172:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97985f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+57:3655:x1871938510809216:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.471893:0:14172:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29519 count 1 00010000:00010000:2.0:1785220582.471908:0:14172:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471921:0:14172:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078dc9338/0xf62499ab92793753 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc38 expref: 25 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471929:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x734f:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.471932:0:14172:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.471936:0:14172:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471957:0:14172:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078dc9338/0xf62499ab92793753 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc38 expref: 25 pid: 14163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.471967:0:14172:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078dc9338/0xf62499ab92793753 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc38 expref: 25 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.471977:0:14172:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510809344:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.472022:0:14172:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.474734:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.474919:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.474948:0:13866:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa61180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:13866:1871938510807552:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.474958:0:13837:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:13837:1871938510808576:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.474977:0:13866:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.474977:0:13837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.475030:0:8752:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8752:1871938510808704:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.475044:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.475197:0:14162:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_024:lustre-OST0001_UUID:14162:1871938510808832:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.475211:0:14162:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.475260:0:14266:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a706300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_086:lustre-OST0001_UUID:14266:1871938510808960:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.475273:0:14266:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.475299:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.475329:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.475348:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.475368:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012160 00000100:00100000:3.0:1785220582.475373:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012288 00000100:00100000:0.0:1785220582.475374:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012160 00000100:00100000:3.0:1785220582.475376:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012288 00010000:00010000:0.0:1785220582.475404:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000170925e5/0xf62499ab927936f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbd6 expref: 27 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.475405:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000064c7d507/0xf62499ab9279370d lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf2 expref: 27 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.475431:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012416 00000100:00100000:0.0:1785220582.475434:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012416 00010000:00010000:0.0:1785220582.475445:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000740c1176/0xf62499ab92793737 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc1c expref: 28 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.475836:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012672 00000100:00100000:3.0:1785220582.475841:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012672 00010000:00010000:3.0:1785220582.475857:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000348e48b0/0xf62499ab92793729 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc0e expref: 29 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.475877:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012544 00000100:00100000:3.0:1785220582.475880:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012544 00010000:00010000:3.0:1785220582.475891:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000076ced996/0xf62499ab92793745 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc2a expref: 30 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.475914:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100821 00000100:00100000:3.0:1785220582.475922:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+30:2407:x1871938520012160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.475936:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.475941:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000170925e5/0xf62499ab927936f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbd6 expref: 30 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.475944:0:14177:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_038:lustre-OST0001_UUID:14177:1871938510808064:192.168.201.51@tcp:104: 00010000:00010000:3.0:1785220582.475952:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000170925e5/0xf62499ab927936f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbd6 expref: 30 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.475959:0:14177:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.475975:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000170925e5/0xf62499ab927936f1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbd6 expref: 30 pid: 13866 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.475979:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100822 00000100:00100000:1.0:1785220582.475987:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+30:2406:x1871938520012288:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.475993:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.475995:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7348:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.475998:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000064c7d507/0xf62499ab9279370d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf2 expref: 29 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476000:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7348:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.476007:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476007:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000064c7d507/0xf62499ab9279370d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf2 expref: 29 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476017:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476020:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000064c7d507/0xf62499ab9279370d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbf2 expref: 29 pid: 14162 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476030:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.476033:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.476036:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476041:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.476044:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476058:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.476122:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+28:2407:x1871938520012160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 199us (1396us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.476129:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100821 00000100:00100000:3.0:1785220582.476147:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100823 00000100:00100000:3.0:1785220582.476155:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+27:2405:x1871938520012416:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.476163:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.476168:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000740c1176/0xf62499ab92793737 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc1c expref: 27 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476178:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000740c1176/0xf62499ab92793737 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc1c expref: 27 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476190:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000740c1176/0xf62499ab92793737 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc1c expref: 27 pid: 13819 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476206:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.476208:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.476212:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476220:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.476238:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220582.476387:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+26:2405:x1871938520012416:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 232us (1086us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.476388:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+26:2406:x1871938520012288:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 400us (1467us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.476395:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100823 00000100:00100000:1.0:1785220582.476424:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100822 00000100:00100000:1.0:1785220582.476441:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100824 00000100:00100000:1.0:1785220582.476448:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+24:2405:x1871938520012672:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.476455:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.476460:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000348e48b0/0xf62499ab92793729 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc0e expref: 24 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476469:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000348e48b0/0xf62499ab92793729 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc0e expref: 24 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476482:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000348e48b0/0xf62499ab92793729 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc0e expref: 24 pid: 13909 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476491:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.476494:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.476498:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476505:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476522:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.476541:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+23:2405:x1871938520012672:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 94us (1211us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.476547:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100824 00000100:00100000:1.0:1785220582.476561:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100825 00000100:00100000:1.0:1785220582.476567:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+22:2405:x1871938520012544:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.476574:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.476579:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000076ced996/0xf62499ab92793745 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc2a expref: 22 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476588:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000076ced996/0xf62499ab92793745 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc2a expref: 22 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476600:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000076ced996/0xf62499ab92793745 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc2a expref: 22 pid: 14261 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476621:0:13866:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476631:0:13866:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476640:0:13866:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476647:0:13866:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476663:0:13866:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.476684:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.476687:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.476691:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476699:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476715:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.476733:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+21:2405:x1871938520012544:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 165us (1383us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.476738:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100825 00010000:00010000:1.0:1785220582.476766:0:14177:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476779:0:14177:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476786:0:14177:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476793:0:14177:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.476805:0:14177:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.476843:0:13866:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97984a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+58:3655:x1871938510807424:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9828us (10195us total) trans 4295070282 rc 0/0 00000100:00100000:2.0:1785220582.476851:0:13866:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284888 00010000:00010000:2.0:1785220582.476873:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1785220582.476879:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476887:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92200 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.476891:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476905:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476917:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476924:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2afd51f/0xf62499ab927a5cce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7348:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13866 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476940:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1785220582.476944:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.476949:0:14177:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_038:lustre-MDT0000-mdtlov_UUID+58:3652:x1871938510807808:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9000us (9465us total) trans 4295070283 rc 0/0 00010000:00010000:2.0:1785220582.476950:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:2.0:1785220582.476953:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.476955:0:14177:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284890 00010000:00010000:2.0:1785220582.476976:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476986:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.476993:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f496ff4c/0xf62499ab927a5cd5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14177 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.477011:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.477013:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.477014:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.477029:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477037:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477044:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477050:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477140:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.477219:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8f725f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+58:3652:x1871938510808448:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7015us (7297us total) trans 4295070284 rc 0/0 00000100:00100000:0.0:1785220582.477226:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284892 00010000:00010000:0.0:1785220582.477244:0:14162:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477252:0:14162:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477259:0:14162:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477266:0:14162:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477330:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.477336:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477344:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.477347:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477355:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477361:0:13837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477366:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477369:0:13837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477374:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000083a28e37/0xf62499ab927a5cea lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477378:0:13837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477384:0:13837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477389:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.477392:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477398:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:3.0:1785220582.477399:0:13837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220582.477401:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477408:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477417:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477425:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.477428:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477435:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3800 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.477438:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477450:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477459:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.477466:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f5b9f9dc/0xf62499ab927a5ce3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.477471:0:13837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+58:3652:x1871938510808320:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7519us (7655us total) trans 4295070285 rc 0/0 00000100:00100000:3.0:1785220582.477477:0:13837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284891 00010000:00010000:1.0:1785220582.477478:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.477480:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.477481:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220582.477572:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.477595:0:13834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012800 00000100:00100000:3.0:1785220582.477601:0:13834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012800 00010000:00010000:3.0:1785220582.477621:0:13834:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001ff5b905/0xf62499ab9279371b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc00 expref: 21 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.477695:0:13834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100826 00000100:00100000:3.0:1785220582.477704:0:13834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97984700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+21:2405:x1871938520012800:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220582.477714:0:13834:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220582.477719:0:13834:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001ff5b905/0xf62499ab9279371b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc00 expref: 21 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477729:0:13834:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001ff5b905/0xf62499ab9279371b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc00 expref: 21 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477742:0:13834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001ff5b905/0xf62499ab9279371b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc00 expref: 21 pid: 14200 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477764:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.477768:0:13834:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.477772:0:13834:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477787:0:13834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.477806:0:13834:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.477812:0:14162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1785220582.477825:0:14162:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5cf1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14162 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.477828:0:13834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97984700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2405:x1871938520012800:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 126us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.477834:0:13834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100826 00000100:00100000:0.0:1785220582.477908:0:14162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97985180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_024:lustre-MDT0000-mdtlov_UUID+58:3655:x1871938510808192:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7144us (7295us total) trans 4295070286 rc 0/0 00000100:00100000:0.0:1785220582.477915:0:14162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284893 00010000:00010000:0.0:1785220582.477934:0:14266:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.477939:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:0.0:1785220582.477943:0:14266:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477952:0:14266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477964:0:14266:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.477976:0:14266:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1785220582.477977:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.478040:0:14266:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97986300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_086:lustre-MDT0000-mdtlov_UUID+58:3655:x1871938510807936:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7113us (7371us total) trans 4295070287 rc 0/0 00000100:00100000:0.0:1785220582.478046:0:14266:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284894 00000004:00080000:3.0:1785220582.478054:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa63480/2, rc 0, transno 4295070287 00000100:00100000:0.0:1785220582.478063:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520012928 00000100:00100000:3.0:1785220582.478065:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa63480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510807936:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.478066:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520012928 00000004:00080000:3.0:1785220582.478076:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa61500/2, rc 0, transno 4295070286 00000100:00100000:3.0:1785220582.478080:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa61500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510808192:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.478080:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cd1b4aac/0xf62499ab927936ff lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbe4 expref: 20 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.478310:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013056 00000100:00100000:0.0:1785220582.478314:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013056 00010000:00010000:0.0:1785220582.478329:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078dc9338/0xf62499ab92793753 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc38 expref: 21 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.478349:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100827 00000100:00100000:0.0:1785220582.478356:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac543c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+21:2407:x1871938520012928:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.478364:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.478369:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cd1b4aac/0xf62499ab927936ff lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbe4 expref: 21 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.478378:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cd1b4aac/0xf62499ab927936ff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbe4 expref: 21 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.478395:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7349:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.478399:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7349:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.478403:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.478411:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.478425:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:3.0:1785220582.478468:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa63b80/2, rc 0, transno 4295070282 00000100:00100000:3.0:1785220582.478474:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa63b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510807424:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.478770:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.478777:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.478785:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.478788:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.478796:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.478806:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.478873:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d3c906d4/0xf62499ab927a5cf8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.478892:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.478894:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.478895:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.478903:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.478955:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac543c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+21:2407:x1871938520012928:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 598us (1015us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.478963:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100827 00000100:00100000:0.0:1785220582.479001:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100828 00000100:00100000:0.0:1785220582.479009:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97985880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2406:x1871938520013056:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.479017:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.479022:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078dc9338/0xf62499ab92793753 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc38 expref: 20 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.479032:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078dc9338/0xf62499ab92793753 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc38 expref: 20 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.479045:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.479048:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x734f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.479052:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.479060:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.479071:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.479103:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97985880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2406:x1871938520013056:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 95us (1125us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.479109:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100828 00010000:00010000:1.0:1785220582.479456:0:14249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd1b4aac/0xf62499ab927936ff lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fbe4 expref: 19 pid: 14248 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.479470:0:14249:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_075:lustre-OST0001_UUID:14249:1871938510809088:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220582.479486:0:14249:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.479495:0:14249:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.479502:0:14249:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.479510:0:14249:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.479517:0:14249:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.479528:0:14249:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1785220582.479696:0:14249:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8fa62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_075:lustre-MDT0000-mdtlov_UUID+58:3652:x1871938510807680:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11786us (12356us total) trans 4295070288 rc 0/0 00000100:00100000:1.0:1785220582.479703:0:14249:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284889 00010000:00010000:0.0:1785220582.480331:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220582.480337:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.480347:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497400 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.480351:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.480360:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.480374:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.480381:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d89f23d5/0xf62499ab927a5cdc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7349:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14249 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.480401:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.480403:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220582.480405:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1785220582.480453:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa60e00/2, rc 0, transno 4295070284 00000100:00100000:0.0:1785220582.480459:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510808448:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.480469:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa60a80/2, rc 0, transno 4295070285 00000100:00100000:0.0:1785220582.480473:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510808320:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.480481:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa60700/2, rc 0, transno 4295070283 00000100:00100000:0.0:1785220582.480484:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510807808:0@lo:6:osp-syn-1-0.0 00000004:00080000:0.0:1785220582.480491:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d8fa60380/2, rc 0, transno 4295070288 00000100:00100000:0.0:1785220582.480494:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8fa60380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510807680:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1785220582.480769:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.480794:0:14172:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078dc9338/0xf62499ab92793753 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc38 expref: 18 pid: 14163 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.480804:0:14172:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_033:lustre-OST0001_UUID:14172:1871938510809344:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.480821:0:14172:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.480830:0:14172:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.480837:0:14172:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.480846:0:14172:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.480855:0:14172:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.480913:0:14172:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1785220582.481027:0:14172:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97985f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_033:lustre-MDT0000-mdtlov_UUID+58:3655:x1871938510809216:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9147us (9254us total) trans 4295070289 rc 0/0 00000100:00100000:2.0:1785220582.481035:0:14172:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284895 00000004:00080000:3.0:1785220582.481050:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d921f8a80/2, rc 0, transno 4295070289 00000100:00100000:3.0:1785220582.481056:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809216:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.481076:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.481082:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481209:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.481214:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481227:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481239:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.481242:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97984e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809472:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.481247:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aed73445/0xf62499ab927a5cff lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x734f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14172 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481263:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.481266:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.481268:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220582.481277:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1785220582.481309:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1785220582.481321:0:14207:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510809472 00000100:00100000:2.0:1785220582.481327:0:14207:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510809472 00000100:00100000:2.0:1785220582.481361:0:14207:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284896 00000100:00100000:0.0:1785220582.481379:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.481385:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97987800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809856:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.481406:0:14207:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+58:3655:x1871938510809472:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.481408:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:2.0:1785220582.481432:0:14207:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29520 count 1 00010000:00010000:2.0:1785220582.481451:0:14207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.481466:0:14207:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000025530976/0xf62499ab92793761 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc46 expref: 17 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.481475:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7350:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.481479:0:14207:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7350:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.481484:0:14207:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.481506:0:14207:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000025530976/0xf62499ab92793761 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc46 expref: 17 pid: 13818 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.481519:0:14207:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000025530976/0xf62499ab92793761 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc46 expref: 17 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.481530:0:14207:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510810112:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.481530:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97986a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809984:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.481572:0:14189:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510809856 00000100:00100000:1.0:1785220582.481576:0:14189:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510809856 00000100:00100000:1.0:1785220582.481602:0:14189:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284897 00000100:00100000:3.0:1785220582.481615:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97984700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510809600:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.481691:0:14189:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+59:3655:x1871938510809856:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1785220582.481711:0:14189:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29523 count 1 00010000:00010000:1.0:1785220582.481731:0:14189:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481756:0:14189:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000595f974b/0xf62499ab9279378b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc70 expref: 17 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481769:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7353:0x0].0x0 since=17s v=(0 0) 00010000:00010000:1.0:1785220582.481775:0:14189:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7353:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.481782:0:14189:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481811:0:14189:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000595f974b/0xf62499ab9279378b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc70 expref: 17 pid: 14195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.481824:0:14189:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000595f974b/0xf62499ab9279378b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc70 expref: 17 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.481836:0:14189:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510810624:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.481891:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.481902:0:14207:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9a4d5600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.481909:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220582.481915:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a5ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510810496:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.481916:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510809984 00000100:00100000:2.0:1785220582.481919:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510809984 00000100:00100000:0.0:1785220582.481936:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.481946:0:13819:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510810496 00000100:00100000:2.0:1785220582.481949:0:13819:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510810496 00000100:00100000:2.0:1785220582.481969:0:13819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284898 00000100:00100000:2.0:1785220582.481975:0:13819:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+61:3655:x1871938510809984:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.481979:0:14205:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284899 00002000:00080000:2.0:1785220582.481986:0:13819:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29524 count 1 00000100:00100000:0.0:1785220582.481986:0:14205:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+61:3655:x1871938510810496:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.481996:0:14205:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29527 count 1 00010000:00010000:2.0:1785220582.481997:0:13819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482007:0:14205:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.482008:0:13819:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000df5ac4e5/0xf62499ab92793799 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc7e expref: 17 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.482016:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7354:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.482019:0:13819:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7354:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.482022:0:13819:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482024:0:14205:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dfa351df/0xf62499ab927937c3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca8 expref: 17 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482031:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7357:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.482034:0:14205:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7357:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.482037:0:14205:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.482040:0:13819:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000df5ac4e5/0xf62499ab92793799 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc7e expref: 17 pid: 14266 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.482050:0:13819:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000df5ac4e5/0xf62499ab92793799 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc7e expref: 17 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482052:0:14205:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dfa351df/0xf62499ab927937c3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca8 expref: 17 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.482061:0:13819:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510810752:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.482062:0:14205:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dfa351df/0xf62499ab927937c3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca8 expref: 17 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.482071:0:14205:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510810880:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.482082:0:13819:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eaccaac00 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.482129:0:14205:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.482188:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510809728:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.482217:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.482241:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a5ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510810368:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1785220582.482248:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510809728 00000100:00100000:0.0:1785220582.482252:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510809728 00000100:00100000:2.0:1785220582.482259:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220582.482275:0:14169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510810368 00000100:00100000:0.0:1785220582.482278:0:14169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510810368 00000100:00100000:0.0:1785220582.482296:0:14169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284900 00000100:00100000:0.0:1785220582.482302:0:14169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+63:3653:x1871938510809728:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.482313:0:14169:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29522 count 1 00010000:00010000:0.0:1785220582.482328:0:14169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482349:0:14169:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000019833a31/0xf62499ab9279377d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc62 expref: 17 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482356:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7352:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.482358:0:14169:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7352:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.482362:0:14169:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482380:0:14169:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000019833a31/0xf62499ab9279377d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc62 expref: 17 pid: 14168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482389:0:14169:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000019833a31/0xf62499ab9279377d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc62 expref: 17 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.482398:0:14169:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510811008:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.482416:0:14169:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.482430:0:14203:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284901 00000100:00100000:0.0:1785220582.482436:0:14203:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+63:3653:x1871938510810368:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.482445:0:14203:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29526 count 1 00010000:00010000:0.0:1785220582.482453:0:14203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.482486:0:14189:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00010000:00010000:0.0:1785220582.482789:0:14203:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000346afe44/0xf62499ab927937b5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc9a expref: 17 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482799:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7356:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.482802:0:14203:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7356:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.482805:0:14203:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482824:0:14203:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000346afe44/0xf62499ab927937b5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc9a expref: 17 pid: 14190 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.482832:0:14203:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000346afe44/0xf62499ab927937b5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc9a expref: 17 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.482841:0:14203:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510811136:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.482863:0:14203:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.483154:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220582.483181:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a5efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510810240:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.483205:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.483290:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510809600 00000100:00100000:1.0:1785220582.483296:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510809600 00000100:00100000:1.0:1785220582.483398:0:14257:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510810240 00000100:00100000:1.0:1785220582.483402:0:14257:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510810240 00000100:00100000:2.0:1785220582.483432:0:13820:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284902 00000100:00100000:2.0:1785220582.483439:0:13820:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+65:3652:x1871938510809600:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1785220582.483455:0:13820:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29521 count 1 00010000:00010000:2.0:1785220582.483542:0:13820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.483556:0:13820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c7614a53/0xf62499ab9279376f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc54 expref: 17 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.483564:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7351:0x0].0x0 since=17s v=(0 0) 00010000:00010000:2.0:1785220582.483567:0:13820:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7351:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.483571:0:13820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.483595:0:13820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c7614a53/0xf62499ab9279376f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc54 expref: 17 pid: 13869 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.483616:0:13820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c7614a53/0xf62499ab9279376f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc54 expref: 17 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.483632:0:13820:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510811264:192.168.201.51@tcp:104: 00000100:00100000:2.0:1785220582.483664:0:13820:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a39ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1785220582.483815:0:14178:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284903 00000100:00100000:0.0:1785220582.483824:0:14178:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+65:3652:x1871938510810240:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1785220582.483836:0:14178:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:29525 count 1 00010000:00010000:0.0:1785220582.483849:0:14178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.483859:0:14178:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000055fb0d0/0xf62499ab927937a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc8c expref: 17 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.483866:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x7355:0x0].0x0 since=17s v=(0 0) 00010000:00010000:0.0:1785220582.483869:0:14178:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7355:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.483873:0:14178:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.483892:0:14178:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000055fb0d0/0xf62499ab927937a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc8c expref: 17 pid: 14161 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.483902:0:14178:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000055fb0d0/0xf62499ab927937a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc8c expref: 17 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.483911:0:14178:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510811392:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.483936:0:14178:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220582.486848:0:14207:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_063:lustre-OST0001_UUID:14207:1871938510810112:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.486879:0:14207:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.486895:0:14169:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_030:lustre-OST0001_UUID:14169:1871938510811008:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.486912:0:14169:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.486972:0:14205:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_061:lustre-OST0001_UUID:14205:1871938510810880:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.486976:0:14203:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_059:lustre-OST0001_UUID:14203:1871938510811136:192.168.201.51@tcp:104: 00010000:00010000:0.0:1785220582.486987:0:14203:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.486988:0:14205:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.487019:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.487047:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220582.487055:0:14189:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_046:lustre-OST0001_UUID:14189:1871938510810624:192.168.201.51@tcp:104: 00000100:00100000:0.0:1785220582.487067:0:8456:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013184 00010000:00010000:2.0:1785220582.487069:0:14189:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.487072:0:8456:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013184 00000100:00100000:1.0:1785220582.487127:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013440 00000100:00100000:1.0:1785220582.487131:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013440 00010000:00010000:1.0:1785220582.487157:0:14229:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dfa351df/0xf62499ab927937c3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca8 expref: 19 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.487187:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100829 00000100:00100000:1.0:1785220582.487197:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+19:2407:x1871938520013440:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.487211:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.487217:0:14229:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dfa351df/0xf62499ab927937c3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca8 expref: 19 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.487228:0:14229:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dfa351df/0xf62499ab927937c3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca8 expref: 19 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.487228:0:8456:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000025530976/0xf62499ab92793761 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc46 expref: 19 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.487252:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dfa351df/0xf62499ab927937c3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fca8 expref: 19 pid: 8442 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.487257:0:8456:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100830 00000100:00100000:0.0:1785220582.487262:0:8456:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+18:2407:x1871938520013184:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.487268:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7357:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.487269:0:8456:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.487272:0:14229:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7357:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.487274:0:8456:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000025530976/0xf62499ab92793761 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc46 expref: 18 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.487278:0:14229:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.487284:0:8456:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000025530976/0xf62499ab92793761 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc46 expref: 18 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.487293:0:14229:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.487297:0:8456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000025530976/0xf62499ab92793761 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc46 expref: 18 pid: 13818 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.487306:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7350:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.487308:0:8456:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7350:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.487312:0:8456:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.487314:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.487319:0:8456:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.487332:0:8456:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220582.487355:0:8456:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+17:2407:x1871938520013184:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 92us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.487363:0:8456:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100830 00010000:00010000:2.0:1785220582.487384:0:14207:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.487393:0:14207:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.487402:0:14207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.487410:0:14207:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.487420:0:13819:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:13819:1871938510810752:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.487426:0:14207:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1785220582.487434:0:13819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.487594:0:14207:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_063:lustre-MDT0000-mdtlov_UUID+66:3655:x1871938510809472:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6224us (6315us total) trans 4295070290 rc 0/0 00000100:00100000:2.0:1785220582.487602:0:14207:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284896 00000004:00080000:3.0:1785220582.487644:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97984e00/2, rc 0, transno 4295070290 00000100:00100000:3.0:1785220582.487651:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97984e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809472:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1785220582.489077:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+16:2407:x1871938520013440:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1874us (2022us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.489104:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100829 00010000:00010000:1.0:1785220582.489139:0:14205:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489150:0:14205:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489160:0:14205:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489168:0:14205:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489190:0:14205:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1785220582.489336:0:14205:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_061:lustre-MDT0000-mdtlov_UUID+66:3655:x1871938510810496:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7350us (7397us total) trans 4295070291 rc 0/0 00000100:00100000:1.0:1785220582.489344:0:14205:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284899 00010000:00010000:1.0:1785220582.489365:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.489370:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489379:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.489383:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489392:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489411:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489418:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000059cef3a/0xf62499ab927a5d06 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7350:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.489431:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220582.489438:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.489441:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489447:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3c00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.489450:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489456:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489465:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489473:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fda89c91/0xf62499ab927a5d1b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7357:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14205 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.489487:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.489489:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.489491:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220582.489787:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220582.489810:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220582.489811:0:13835:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013696 00000100:00100000:1.0:1785220582.489841:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.489864:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220582.489883:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.489890:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013312 00000100:00100000:0.0:1785220582.489895:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013312 00010000:00010000:0.0:1785220582.489924:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000df5ac4e5/0xf62499ab92793799 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc7e expref: 16 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.489940:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220582.489947:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013568 00000100:00100000:0.0:1785220582.489950:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013568 00010000:00010000:0.0:1785220582.489962:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000019833a31/0xf62499ab9279377d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc62 expref: 17 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.489987:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013952 00000100:00100000:0.0:1785220582.489990:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013952 00010000:00010000:0.0:1785220582.490002:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c7614a53/0xf62499ab9279376f lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc54 expref: 18 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.490020:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520013824 00000100:00100000:0.0:1785220582.490022:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013824 00010000:00010000:0.0:1785220582.490033:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000346afe44/0xf62499ab927937b5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc9a expref: 19 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.490049:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014080 00000100:00100000:0.0:1785220582.490052:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014080 00010000:00010000:0.0:1785220582.490063:0:14235:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000055fb0d0/0xf62499ab927937a7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc8c expref: 20 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.490083:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100831 00000100:00100000:2.0:1785220582.490247:0:13820:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:13820:1871938510811264:192.168.201.51@tcp:104: 00010000:00010000:2.0:1785220582.490268:0:13820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.490457:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2407:x1871938520013312:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.490473:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.490479:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000df5ac4e5/0xf62499ab92793799 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc7e expref: 20 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490490:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000df5ac4e5/0xf62499ab92793799 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc7e expref: 20 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.490494:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100832 00000100:00100000:1.0:1785220582.490502:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2407:x1871938520013568:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1785220582.490507:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df5ac4e5/0xf62499ab92793799 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc7e expref: 20 pid: 14266 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.490510:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.490514:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000019833a31/0xf62499ab9279377d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc62 expref: 20 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490520:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7354:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.490523:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000019833a31/0xf62499ab9279377d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc62 expref: 19 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490524:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7354:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.490529:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.490534:0:13817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019833a31/0xf62499ab9279377d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc62 expref: 19 pid: 14168 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490539:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.490542:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7352:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.490546:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7352:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.490549:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.490556:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490563:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220582.490572:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.490595:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100833 00000100:00100000:0.0:1785220582.490601:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+18:2407:x1871938520013312:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 148us (792us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.490603:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ea4e13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+18:2407:x1871938520013952:12345-192.168.201.51@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1785220582.490608:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100831 00010000:00010000:2.0:1785220582.490621:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220582.490623:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100834 00010000:00010000:2.0:1785220582.490626:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c7614a53/0xf62499ab9279376f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc54 expref: 17 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.490629:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+17:2407:x1871938520013824:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220582.490637:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c7614a53/0xf62499ab9279376f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc54 expref: 17 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490637:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220582.490642:0:14235:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000346afe44/0xf62499ab927937b5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc9a expref: 17 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.490650:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c7614a53/0xf62499ab9279376f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc54 expref: 17 pid: 13869 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490650:0:14235:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000346afe44/0xf62499ab927937b5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc9a expref: 17 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.490661:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7351:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.490662:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000346afe44/0xf62499ab927937b5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc9a expref: 16 pid: 14190 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.490664:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7351:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.490668:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490672:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7356:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1785220582.490674:0:14235:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7356:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1785220582.490676:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490678:0:14235:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.490685:0:14235:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.490692:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220582.490700:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220582.490790:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+15:2407:x1871938520013568:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 288us (947us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.490791:0:13835:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520013696 00000100:00100000:1.0:1785220582.490798:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100832 00010000:00010000:3.0:1785220582.490812:0:13835:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000595f974b/0xf62499ab9279378b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc70 expref: 15 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.490812:0:13817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100835 00000100:00100000:1.0:1785220582.490818:0:13817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+15:2407:x1871938520014080:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.490825:0:13817:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.490830:0:13817:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000055fb0d0/0xf62499ab927937a7 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc8c expref: 15 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.490839:0:13817:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000055fb0d0/0xf62499ab927937a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc8c expref: 15 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220582.490845:0:13835:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100836 00000100:00100000:3.0:1785220582.490851:0:13835:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+15:2407:x1871938520013696:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1785220582.490851:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7355:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1785220582.490854:0:13817:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7355:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.490858:0:13835:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220582.490858:0:13817:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.490862:0:13835:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000595f974b/0xf62499ab9279378b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc70 expref: 15 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.490866:0:13817:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.490872:0:13835:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000595f974b/0xf62499ab9279378b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc70 expref: 15 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.490878:0:13817:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220582.490884:0:13835:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000595f974b/0xf62499ab9279378b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc70 expref: 15 pid: 14195 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.490893:0:13817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+14:2407:x1871938520014080:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 76us (953us total) trans 0 rc 0/0 00010000:00010000:3.0:1785220582.490894:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7353:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1785220582.490897:0:13835:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x7353:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1785220582.490900:0:13817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100835 00010000:00010000:3.0:1785220582.490901:0:13835:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.490913:0:13835:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220582.490938:0:13835:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220582.490948:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ea4e13850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+13:2407:x1871938520013952:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 346us (1083us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220582.490955:0:13835:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+13:2407:x1871938520013696:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 104us (1524us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220582.490955:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100833 00000100:00100000:3.0:1785220582.490960:0:13835:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100836 00010000:00010000:2.0:1785220582.490973:0:13820:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.490981:0:13820:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.490989:0:13820:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.490996:0:13820:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220582.491008:0:13820:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1785220582.491019:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a5ef100/2, rc 0, transno 4295070291 00000100:00100000:3.0:1785220582.491026:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a5ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510810496:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1785220582.491128:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.491134:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491142:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.491146:0:13859:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491154:0:13859:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491166:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491173:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b499a259/0xf62499ab927a5d30 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7351:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491190:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.491192:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.491194:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.491204:0:14189:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491212:0:14189:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491220:0:14189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491227:0:14189:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491247:0:14189:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1785220582.491276:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+11:2407:x1871938520013824:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 646us (1391us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220582.491283:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100834 00010000:00010000:0.0:1785220582.491315:0:14178:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000055fb0d0/0xf62499ab927937a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fc8c expref: 10 pid: 14161 timeout: 1154 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.491326:0:14178:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_039:lustre-OST0001_UUID:14178:1871938510811392:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220582.491339:0:14189:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_046:lustre-MDT0000-mdtlov_UUID+67:3655:x1871938510809856:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9655us (9928us total) trans 4295070293 rc 0/0 00010000:00010000:0.0:1785220582.491339:0:14178:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220582.491347:0:14189:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284897 00010000:00010000:0.0:1785220582.491348:0:14178:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491354:0:14178:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491361:0:14178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491368:0:14178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491386:0:14178:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1785220582.491437:0:13820:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+67:3652:x1871938510809600:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7998us (8282us total) trans 4295070292 rc 0/0 00000100:00100000:2.0:1785220582.491444:0:13820:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284902 00000100:00100000:0.0:1785220582.491456:0:14178:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_039:lustre-MDT0000-mdtlov_UUID+66:3652:x1871938510810240:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7635us (8250us total) trans 4295070294 rc 0/0 00000100:00100000:0.0:1785220582.491462:0:14178:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284903 00000004:00080000:2.0:1785220582.491472:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97984700/2, rc 0, transno 4295070292 00000100:00100000:2.0:1785220582.491476:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97984700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510809600:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.491479:0:14203:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220582.491486:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a5efb80/2, rc 0, transno 4295070294 00010000:00010000:0.0:1785220582.491488:0:14203:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220582.491489:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a5efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510810240:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1785220582.491496:0:14203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491508:0:14203:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491528:0:14203:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1785220582.491598:0:14203:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_059:lustre-MDT0000-mdtlov_UUID+66:3653:x1871938510810368:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9162us (9337us total) trans 4295070295 rc 0/0 00000100:00100000:0.0:1785220582.491604:0:14203:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284901 00010000:00010000:0.0:1785220582.491625:0:14169:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491634:0:14169:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491642:0:14169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491649:0:14169:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491660:0:14169:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:1.0:1785220582.491703:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1785220582.491708:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491716:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.491719:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491727:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.491729:0:14169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+66:3653:x1871938510809728:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9427us (9512us total) trans 4295070296 rc 0/0 00000100:00100000:0.0:1785220582.491735:0:14169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284900 00010000:00010000:1.0:1785220582.491738:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491757:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000061322d3c/0xf62499ab927a5d0d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7353:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491769:0:13819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491773:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.491777:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491784:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2000 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:0.0:1785220582.491784:0:13819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491787:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491793:0:13819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491795:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491801:0:13819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491805:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220582.491809:0:13819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1785220582.491812:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e74303e/0xf62499ab927a5d37 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7355:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491822:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1785220582.491826:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491832:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3600 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.491835:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491843:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491852:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491859:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0789cf9/0xf62499ab927a5d29 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7356:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14203 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491872:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220582.491875:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.491879:0:13819:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+66:3655:x1871938510809984:12345-0@lo:6:osp-syn-1-0.0 Request processed in 9905us (9989us total) trans 4295070297 rc 0/0 00010000:00010000:1.0:1785220582.491882:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2e00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.491885:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220582.491886:0:13819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284898 00010000:00010000:1.0:1785220582.491892:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491901:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491908:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f67db09c/0xf62499ab927a5d22 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7352:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14169 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491919:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220582.491921:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491927:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92a00 already unused, calling callback (ffffffffc0e5c310) 00010000:00010000:1.0:1785220582.491930:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491936:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491946:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491954:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000be5f092/0xf62499ab927a5d14 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x7354:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220582.491966:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.491967:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.491969:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220582.491975:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220582.492010:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1785220582.492029:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97987b80/2, rc 0, transno 4295070296 00000100:00100000:2.0:1785220582.492034:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510809728:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1785220582.492043:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d9a5ed500/2, rc 0, transno 4295070295 00000100:00100000:2.0:1785220582.492046:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a5ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510810368:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1785220582.492051:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1785220582.492067:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97986a00/2, rc 0, transno 4295070297 00000100:00100000:3.0:1785220582.492072:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97986a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809984:0@lo:6:osp-syn-1-0.0 00000004:00080000:3.0:1785220582.492080:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960d97987800/2, rc 0, transno 4295070293 00000100:00100000:3.0:1785220582.492083:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d97987800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510809856:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1785220582.635228:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1785220582.635253:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510811520:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1785220582.635327:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220582.635458:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510811520 00000100:00100000:1.0:1785220582.635466:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510811520 00000100:00100000:1.0:1785220582.635525:0:13815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12555 00000100:00100000:1.0:1785220582.635534:0:13815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1871938510811520:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1785220582.635572:0:13815:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1440874496 avail 1294635008 00000100:00100000:1.0:1785220582.635620:0:13815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1871938510811520:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220582.635626:0:13815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12555 00000100:00100000:2.0:1785220582.635863:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510811520:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1785220582.712969:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220583.486268:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220583.486311:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220583.486345:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220583.486374:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220583.486391:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220583.486416:0:13872:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014336 00000100:00100000:1.0:1785220583.486427:0:13872:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014336 00000100:00100000:1.0:1785220583.486481:0:13872:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014464 00000100:00100000:1.0:1785220583.486485:0:13872:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014464 00000100:00100000:1.0:1785220583.486503:0:13872:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014848 00000100:00100000:1.0:1785220583.486505:0:13872:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014848 00000100:00100000:1.0:1785220583.486526:0:13872:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1350 00000100:00100000:1.0:1785220583.486535:0:13872:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:5644b1e7-55e7-4421-a7e4-714e45fd656f+704:2404:x1871938520014336:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:1.0:1785220583.486637:0:13872:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:5644b1e7-55e7-4421-a7e4-714e45fd656f+704:2404:x1871938520014336:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 102us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220583.486639:0:13846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1351 00000100:00100000:1.0:1785220583.486644:0:13872:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1350 00000100:00100000:2.0:1785220583.486648:0:13846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_017:5644b1e7-55e7-4421-a7e4-714e45fd656f+485:2405:x1871938520014464:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:1.0:1785220583.486660:0:13872:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1352 00000100:00100000:1.0:1785220583.486666:0:13872:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2404:x1871938520014848:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:2.0:1785220583.486682:0:13846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_017:5644b1e7-55e7-4421-a7e4-714e45fd656f+485:2405:x1871938520014464:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 36us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220583.486688:0:13846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1351 00000100:00100000:1.0:1785220583.486694:0:13872:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2404:x1871938520014848:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 29us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220583.486699:0:13872:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1352 00000100:00080000:2.0:1785220583.486705:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1785220583.486709:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1785220583.486748:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220583.486817:0:14210:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014592 00000100:00100000:2.0:1785220583.486821:0:14210:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014592 00000100:00100000:1.0:1785220583.486822:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00080000:2.0:1785220583.486836:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1785220583.486851:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220583.486876:0:13857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014976 00000100:00100000:2.0:1785220583.486880:0:13857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014976 00000100:00100000:2.0:1785220583.486906:0:13857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1353 00000100:00100000:2.0:1785220583.486912:0:13857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_020:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+544:2405:x1871938520014976:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:1.0:1785220583.486934:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220583.486946:0:13857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_020:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+544:2405:x1871938520014976:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 34us (95us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220583.486956:0:13857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1353 00000100:00100000:3.0:1785220583.487011:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014720 00000100:00100000:3.0:1785220583.487016:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014720 00000100:00100000:3.0:1785220583.487040:0:14248:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520015104 00000100:00100000:3.0:1785220583.487044:0:14248:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520015104 00000100:00100000:0.0:1785220583.487118:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520014208 00000100:00100000:0.0:1785220583.487124:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520014208 00000100:00100000:0.0:1785220583.487154:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 608 00000100:00100000:0.0:1785220583.487162:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac528450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e3216512-9662-458b-a984-8fbdc3fc5dc2+13:2405:x1871938520014208:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:0.0:1785220583.487192:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac528450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e3216512-9662-458b-a984-8fbdc3fc5dc2+13:2405:x1871938520014208:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 32us (946us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220583.487199:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 608 00000100:00100000:1.0:1785220583.487210:0:13867:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520015232 00000100:00100000:1.0:1785220583.487213:0:13867:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520015232 00000100:00080000:2.0:1785220583.492762:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1785220583.492775:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1785220583.492781:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1785220583.492816:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 428 00000100:00100000:2.0:1785220583.492840:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:2404:x1871938520014592:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:2.0:1785220583.492931:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:2404:x1871938520014592:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 104us (6585us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220583.492939:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 428 00000100:00100000:2.0:1785220583.492964:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 429 00000100:00100000:2.0:1785220583.492970:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:5644b1e7-55e7-4421-a7e4-714e45fd656f+697:2405:x1871938520014720:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:2.0:1785220583.492996:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:5644b1e7-55e7-4421-a7e4-714e45fd656f+697:2405:x1871938520014720:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 26us (6247us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220583.493002:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 429 00000100:00100000:2.0:1785220583.493210:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 430 00000100:00100000:2.0:1785220583.493219:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2404:x1871938520015104:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:2.0:1785220583.493246:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2404:x1871938520015104:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 30us (6424us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220583.493252:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 430 00000100:00100000:2.0:1785220583.493267:0:14210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 431 00000100:00100000:2.0:1785220583.493274:0:14210:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+10:2405:x1871938520015232:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:2.0:1785220583.493297:0:14210:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_066:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+10:2405:x1871938520015232:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 23us (6363us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220583.493306:0:14210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 431 00010000:00010000:1.0:1785220583.721361:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1785220584.745354:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220585.035872:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220585.035971:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220585.036212:0:14276:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520015360 00000100:00100000:0.0:1785220585.036225:0:14276:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520015360 00002000:00100000:0.0:1785220585.036302:0:14276:0:(ofd_dev.c:2709:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_019: refresh rw locks for [0x2c0000401:0x872c:0x0] (0->4) req@ffff960d8add6a00 x1871938520015360/t0(0) o4->5644b1e7-55e7-4421-a7e4-714e45fd656f@192.168.201.51@tcp:446/0 lens 496/0 e 0 to 0 dl 1785220596 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dir_create.sh.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1785220585.036321:0:14276:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff960d8add6a00 with x1871938520015360 ext(0->4) 00002000:00010000:0.0:1785220585.036327:0:14276:0:(ofd_dev.c:2714:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960d8add6a00 00000100:00100000:0.0:1785220585.036357:0:14276:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520015488 00000100:00100000:0.0:1785220585.036361:0:14276:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520015488 00002000:00100000:0.0:1785220585.036377:0:14276:0:(ofd_dev.c:2709:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_019: refresh rw locks for [0x2c0000401:0x8730:0x0] (0->4) req@ffff960d8add4a80 x1871938520015488/t0(0) o4->5644b1e7-55e7-4421-a7e4-714e45fd656f@192.168.201.51@tcp:446/0 lens 496/0 e 0 to 0 dl 1785220596 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dir_create.sh.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1785220585.036385:0:14276:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff960d8add4a80 with x1871938520015488 ext(0->4) 00002000:00010000:0.0:1785220585.036388:0:14276:0:(ofd_dev.c:2714:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960d8add4a80 00000100:00100000:0.0:1785220585.036405:0:14276:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 35492 00000100:00100000:0.0:1785220585.036413:0:14276:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_019:5644b1e7-55e7-4421-a7e4-714e45fd656f+698:2404:x1871938520015360:12345-192.168.201.51@tcp:4:dir_create.sh.0 00000100:00080000:3.0:1785220585.036527:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1785220585.036577:0:14230:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 35493 00000100:00100000:1.0:1785220585.036588:0:14230:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+698:2404:x1871938520015488:12345-192.168.201.51@tcp:4:dir_create.sh.0 00000100:00100000:1.0:1785220585.037490:0:14230:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+702:2404:x1871938520015488:12345-192.168.201.51@tcp:4:dir_create.sh.0 Request processed in 902us (1515us total) trans 4295070298 rc 0/0 00000100:00100000:1.0:1785220585.037501:0:14230:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 35493 00002000:00100000:1.0:1785220585.037509:0:14230:0:(ofd_dev.c:2709:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_015: refresh rw locks for [0x2c0000401:0x8730:0x0] (0->4) req@ffff960d8add4a80 x1871938520015488/t4295070298(0) o4->5644b1e7-55e7-4421-a7e4-714e45fd656f@192.168.201.51@tcp:446/0 lens 496/448 e 0 to 0 dl 1785220596 ref 1 fl Complete:/600/0 rc 0/0 job:'dir_create.sh.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1785220585.037519:0:14230:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff960d8add4a80 with x1871938520015488 ext(0->4) 00002000:00010000:1.0:1785220585.037524:0:14230:0:(ofd_dev.c:2714:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960d8add4a80 00000100:00100000:1.0:1785220585.038177:0:14276:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_019:5644b1e7-55e7-4421-a7e4-714e45fd656f+701:2404:x1871938520015360:12345-192.168.201.51@tcp:4:dir_create.sh.0 Request processed in 1763us (2318us total) trans 4295070299 rc 0/0 00000100:00100000:1.0:1785220585.038186:0:14276:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 35492 00002000:00100000:1.0:1785220585.038194:0:14276:0:(ofd_dev.c:2709:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_019: refresh rw locks for [0x2c0000401:0x872c:0x0] (0->4) req@ffff960d8add6a00 x1871938520015360/t4295070299(0) o4->5644b1e7-55e7-4421-a7e4-714e45fd656f@192.168.201.51@tcp:446/0 lens 496/448 e 0 to 0 dl 1785220596 ref 1 fl Complete:/600/0 rc 0/0 job:'dir_create.sh.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1785220585.038203:0:14276:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff960d8add6a00 with x1871938520015360 ext(0->4) 00002000:00010000:1.0:1785220585.038207:0:14276:0:(ofd_dev.c:2714:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960d8add6a00 00000100:00100000:1.0:1785220585.193288:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510811648:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1785220585.193361:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.193440:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510811648 00000100:00100000:1.0:1785220585.193447:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510811648 00000100:00100000:1.0:1785220585.193498:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1785220585.193509:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+206:3652:x1871938510811648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1785220585.193543:0:8443:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 143360 granted 970222400 pending 0 free 3903991808 avail 3672469504 00000100:00100000:1.0:1785220585.193596:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+206:3652:x1871938510811648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.193603:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1785220585.193650:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1871938510811648:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1785220585.195286:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00020000:0.0:1785220585.705989:0:6516:0:(ldlm_lockd.c:255:expired_lock_main()) ### lock callback timer expired after 100s: evicting client at 192.168.201.51@tcp ns: mdt-lustre-MDT0001_UUID lock: ffff960d97980600/0xf62499ab927272e1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ca63 expref: 544 pid: 13860 timeout: 1057 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1785220585.709542:0:6505:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 368 committed 00000004:00080000:3.0:1785220585.709587:0:6505:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 379 committed 00000020:00080000:0.0:1785220585.739542:0:6516:0:(genops.c:1655:class_fail_export()) disconnecting export ffff960ebd9da800/45383443-d1a2-4f43-8eb5-d11e3b9f90c7 00000020:00000080:0.0:1785220585.739561:0:6516:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0xf62499ab92487c3b 00000020:00010000:0.0:1785220585.739587:0:6516:0:(genops.c:1942:obd_stale_export_put()) Put export ffff960ebd9da800: total 0 00000100:00080000:0.0:1785220585.739593:0:6516:0:(import.c:65:import_set_state_nolock()) ffff960ebd9dc000 : changing import state from FULL to CLOSED 00000004:00080000:0.0:1785220585.741365:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000402:0xf6:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000004:00080000:0.0:1785220585.741488:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000402:0x222:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000004:00080000:0.0:1785220585.741552:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000402:0x4f3:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000004:00080000:0.0:1785220585.741703:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000402:0x6cb:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000020:00010000:3.0:1785220585.742320:0:10345:0:(genops.c:1928:obd_stale_export_get()) Get export ffff960ebd9da800: total 0 00010000:00010000:3.0:1785220585.742385:0:10345:0:(ldlm_lock.c:2652:ldlm_export_cancel_blocked_locks()) Export ffff960ebd9da800, canceled 0 locks, left on hash table 535. 00010000:00010000:3.0:1785220585.742419:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4e3:0x0].0x0 (ffff960ead0e0a00) 00010000:00010000:3.0:1785220585.742423:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742434:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88510400/0xf62499ab925697e0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4e3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa7fcc expref: 544 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742460:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x25:0x0].0x0 (ffff960d8a385300) 00010000:00010000:3.0:1785220585.742463:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.742467:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d90f200/0xf62499ab9260ee6c lrc: 0/0,0 mode: --/PR res: [0x240000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af1373 expref: 543 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742482:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x366:0x0].0x0 (ffff960d8c356d00) 00010000:00010000:3.0:1785220585.742484:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742488:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89de5600/0xf62499ab9252473e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x366:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a91879 expref: 542 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742501:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x29d:0x0].0x0 (ffff960d8d983600) 00010000:00010000:3.0:1785220585.742503:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742507:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f4a400/0xf62499ab92501355 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x29d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a86359 expref: 541 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742523:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x280:0x0].0x0 (ffff960d89fe5a00) 00010000:00010000:3.0:1785220585.742525:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742529:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8effca00/0xf62499ab92500914 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x280:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a85fbd expref: 540 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742544:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xad4:0x0].0x0 (ffff960d8c960c00) 00010000:00010000:3.0:1785220585.742546:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742550:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95695400/0xf62499ab92787f0e lrc: 0/0,0 mode: --/CR res: [0x240000403:0xad4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b853 expref: 539 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742565:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x95f:0x0].0x0 (ffff960d993edf00) 00010000:00010000:3.0:1785220585.742568:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742572:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98aa6000/0xf62499ab926fdc88 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x95f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b4ca42 expref: 538 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742591:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8e3:0x0].0x0 (ffff960d95754700) 00010000:00010000:3.0:1785220585.742593:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742597:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d985f4800/0xf62499ab926b8d7c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8e3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b311e3 expref: 537 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742613:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8be:0x0].0x0 (ffff960d90c25c00) 00010000:00010000:3.0:1785220585.742616:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742620:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94b37000/0xf62499ab926ab4fa lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8be:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b2d2f1 expref: 536 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742636:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2ed:0x0].0x0 (ffff960ead0e0700) 00010000:00010000:3.0:1785220585.742638:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742642:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc38a00/0xf62499ab92510c30 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2ed:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8b3f5 expref: 535 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742657:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x264:0x0].0x0 (ffff960d8c69fc00) 00010000:00010000:3.0:1785220585.742660:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.742663:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d965f5800/0xf62499ab925c8933 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x264:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad0c85 expref: 534 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742676:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x30f:0x0].0x0 (ffff960d8c94bd00) 00010000:00010000:3.0:1785220585.742679:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742683:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c7ea00/0xf62499ab925168d4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x30f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8d1a0 expref: 533 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742717:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x292:0x0].0x0 (ffff960d8a12c800) 00010000:00010000:3.0:1785220585.742720:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742724:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a38c200/0xf62499ab924fe80c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x292:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a853bc expref: 532 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742743:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaa1:0x0].0x0 (ffff960d956af700) 00010000:00010000:3.0:1785220585.742746:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742749:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9754ae00/0xf62499ab927800b3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77c8d expref: 531 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742777:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa7c:0x0].0x0 (ffff960d986bcf00) 00010000:00010000:3.0:1785220585.742779:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742783:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c590e00/0xf62499ab9276c0c7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74176 expref: 530 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742799:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xac1:0x0].0x0 (ffff960d9ada6d00) 00010000:00010000:3.0:1785220585.742801:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742805:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cefde00/0xf62499ab9278469e lrc: 0/0,0 mode: --/CR res: [0x240000403:0xac1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79e13 expref: 529 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742821:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4ec:0x0].0x0 (ffff960d8b1bbd00) 00010000:00010000:3.0:1785220585.742824:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742827:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d900e4e00/0xf62499ab92571412 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4ec:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aaa6bc expref: 528 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742843:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x17e:0x0].0x0 (ffff960eba8e7500) 00010000:00010000:3.0:1785220585.742846:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.742850:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3c200/0xf62499ab924df442 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x17e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7841d expref: 527 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742864:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7e2:0x0].0x0 (ffff960d8b539500) 00010000:00010000:3.0:1785220585.742867:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.742871:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f60f400/0xf62499ab92634b98 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x7e2:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afe431 expref: 526 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742884:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4a9:0x0].0x0 (ffff960d8cc44700) 00010000:00010000:3.0:1785220585.742886:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742890:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cf31000/0xf62499ab925690c4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4a9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa7dfe expref: 525 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742905:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa3f:0x0].0x0 (ffff960d90eb3f00) 00010000:00010000:3.0:1785220585.742908:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742911:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bb35200/0xf62499ab9274eb65 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa3f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b69b13 expref: 524 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742926:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x322:0x0].0x0 (ffff960e8cfeb200) 00010000:00010000:3.0:1785220585.742929:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742933:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e5a00/0xf62499ab9251a0f7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x322:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8e30b expref: 523 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742946:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa45:0x0].0x0 (ffff960d9499f900) 00010000:00010000:3.0:1785220585.742949:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742953:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90173e00/0xf62499ab927515c7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa45:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6a577 expref: 522 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.742974:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x81d:0x0].0x0 (ffff960d985ca600) 00010000:00010000:3.0:1785220585.742976:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.742980:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92380000/0xf62499ab9263e926 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x81d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b00c40 expref: 521 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743009:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3d8:0x0].0x0 (ffff960e87ac1800) 00010000:00010000:3.0:1785220585.743012:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.743016:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87b7d000/0xf62499ab9253a679 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3d8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a98cd9 expref: 520 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743029:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x778:0x0].0x0 (ffff960e828e8500) 00010000:00010000:3.0:1785220585.743032:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.743036:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95613e00/0xf62499ab92604ae8 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x778:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aedd56 expref: 519 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743049:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4fe:0x0].0x0 (ffff960d8ce3e900) 00010000:00010000:3.0:1785220585.743051:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.743055:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87e20000/0xf62499ab9256f9f5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4fe:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa9e96 expref: 518 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743070:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4cb:0x0].0x0 (ffff960d8b1bb000) 00010000:00010000:3.0:1785220585.743072:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.743076:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9036a200/0xf62499ab9256734a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4cb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa7599 expref: 517 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1785220585.743363:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].1240, 440 off891168 00000040:00080000:0.0:1785220585.743536:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].1244, 440 off1070520 00000004:00080000:0.0:1785220585.743544:0:6516:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff960d95b9f9c0 with version 2429 00010000:00010000:3.0:1785220585.743639:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5ce:0x0].0x0 (ffff960d909fa300) 00010000:00010000:3.0:1785220585.743644:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.743650:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9036b400/0xf62499ab9259a02a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x5ce:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab7073 expref: 516 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743666:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x456:0x0].0x0 (ffff960d8c221400) 00010000:00010000:3.0:1785220585.743668:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.743671:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c591200/0xf62499ab9255d139 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x456:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa4189 expref: 515 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743682:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x51e:0x0].0x0 (ffff960d8d983000) 00010000:00010000:3.0:1785220585.743685:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.743697:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ee07e00/0xf62499ab925799c8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x51e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aad187 expref: 514 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743710:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5c9:0x0].0x0 (ffff960d8aee6400) 00010000:00010000:3.0:1785220585.743713:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.743716:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b5bde00/0xf62499ab92589fa7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5c9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab266e expref: 513 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743731:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x482:0x0].0x0 (ffff960d9061c800) 00010000:00010000:3.0:1785220585.743734:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.743737:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfb8c600/0xf62499ab9255a215 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x482:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa325c expref: 512 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.743750:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x333:0x0].0x0 (ffff960d8bde2300) 00010000:00010000:3.0:1785220585.743753:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1785220585.743756:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfb8d400/0xf62499ab9251ff3f lrc: 0/0,0 mode: --/CR res: [0x240000403:0x333:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a90333 expref: 511 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220585.743849:0:7470:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff960ea686e240 version 2429 rpc_version 2429 00000020:00100000:1.0:1785220585.743887:0:7470:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1785220585.743897:0:7470:0:(client.c:807:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff960d8add5c00 x1871938510811904/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1400/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1785220585.743920:0:7470:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add5c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510811904:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1785220585.743988:0:7470:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.744041:0:7470:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d8960c0 going to sleep for 11 seconds 00010000:00010000:3.0:1785220585.744196:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa69:0x0].0x0 (ffff960d98723700) 00010000:00010000:3.0:1785220585.744199:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744204:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9754a600/0xf62499ab9276075c lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa69:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b700b6 expref: 510 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744219:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x31e:0x0].0x0 (ffff960ea4cd7800) 00010000:00010000:3.0:1785220585.744221:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744225:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89ecf200/0xf62499ab92518dda lrc: 0/0,0 mode: --/CR res: [0x240000403:0x31e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8de11 expref: 509 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744239:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8eb:0x0].0x0 (ffff960d98455900) 00010000:00010000:3.0:1785220585.744241:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744245:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8dd86800/0xf62499ab926bbead lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8eb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b31e93 expref: 508 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744255:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x345:0x0].0x0 (ffff960d87ba6d00) 00010000:00010000:3.0:1785220585.744257:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744261:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d136000/0xf62499ab9251ee59 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x345:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8fbbc expref: 507 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744277:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960d956ac600) 00010000:00010000:3.0:1785220585.744279:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.744285:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x38b:0x0].0x0 (ffff960d89d21b00) 00010000:00010000:3.0:1785220585.744287:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744290:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebdb2b400/0xf62499ab9252c522 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x38b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a93ec8 expref: 506 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744308:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4dd:0x0].0x0 (ffff960d8cea4700) 00010000:00010000:3.0:1785220585.744310:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744313:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a039800/0xf62499ab92587b8f lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4dd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1b5b expref: 505 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744327:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4dd:0x0].0x0 (ffff960e9ac51000) 00010000:00010000:3.0:1785220585.744329:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744333:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f9ece00/0xf62499ab92568bae lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4dd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa7c30 expref: 504 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744348:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7e9:0x0].0x0 (ffff960d9795f900) 00010000:00010000:3.0:1785220585.744351:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744354:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c370c00/0xf62499ab92624d4c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7e9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af7717 expref: 503 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744367:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1a4:0x0].0x0 (ffff960e829ca800) 00010000:00010000:3.0:1785220585.744369:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744372:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88513c00/0xf62499ab924d2d6a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1a4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a74707 expref: 502 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744386:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x326:0x0].0x0 (ffff960d8aee6900) 00010000:00010000:3.0:1785220585.744389:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744393:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c7ab600/0xf62499ab9251af91 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x326:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8e883 expref: 501 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744411:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5bf:0x0].0x0 (ffff960d89ff2c00) 00010000:00010000:3.0:1785220585.744415:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744419:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8fe04e00/0xf62499ab925891a0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5bf:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab21b3 expref: 500 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744434:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7aa:0x0].0x0 (ffff960d8dcd2d00) 00010000:00010000:3.0:1785220585.744437:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744441:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95a10a00/0xf62499ab92611929 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7aa:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af2181 expref: 499 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744458:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x975:0x0].0x0 (ffff960d90440300) 00010000:00010000:3.0:1785220585.744461:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.744465:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9586ec00/0xf62499ab9274b3c0 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x975:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b68d83 expref: 498 pid: 13861 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744478:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x76d:0x0].0x0 (ffff960d95755800) 00010000:00010000:3.0:1785220585.744481:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744484:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d978d7800/0xf62499ab925fdd9d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x76d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aec148 expref: 497 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744500:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7e4:0x0].0x0 (ffff960d9065a700) 00010000:00010000:3.0:1785220585.744502:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.744506:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ab62200/0xf62499ab92623f6f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x7e4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af728d expref: 496 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744519:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x971:0x0].0x0 (ffff960d90937900) 00010000:00010000:3.0:1785220585.744521:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744525:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95743a00/0xf62499ab92702bfe lrc: 0/0,0 mode: --/CR res: [0x240000403:0x971:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b4e0b5 expref: 495 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744544:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x366:0x0].0x0 (ffff960d8c356d00) 00010000:00010000:3.0:1785220585.744547:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1785220585.744551:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b5bc000/0xf62499ab9258ea85 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x366:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab3e70 expref: 494 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744568:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8b2:0x0].0x0 (ffff960d909f8600) 00010000:00010000:3.0:1785220585.744571:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744575:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8fab0200/0xf62499ab926aab92 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8b2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b2cf94 expref: 493 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744589:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x405:0x0].0x0 (ffff960d8bba3000) 00010000:00010000:3.0:1785220585.744592:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744595:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c01a600/0xf62499ab9254c851 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x405:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9ee00 expref: 492 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744614:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4c4:0x0].0x0 (ffff960d87ba6100) 00010000:00010000:3.0:1785220585.744617:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744620:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c7aa000/0xf62499ab92564eec lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4c4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa6848 expref: 491 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744636:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x54f:0x0].0x0 (ffff960eacc3e200) 00010000:00010000:3.0:1785220585.744638:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744642:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d900e4a00/0xf62499ab9257eb75 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x54f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aaecf4 expref: 490 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744657:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x67a:0x0].0x0 (ffff960d8d73c700) 00010000:00010000:3.0:1785220585.744660:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744664:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90a7e800/0xf62499ab9259c7fa lrc: 0/0,0 mode: --/CR res: [0x240000403:0x67a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab7cf2 expref: 489 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744678:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x59c:0x0].0x0 (ffff960d87ba4100) 00010000:00010000:3.0:1785220585.744681:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744685:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c590a00/0xf62499ab925865d9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x59c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1503 expref: 488 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744718:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9e3:0x0].0x0 (ffff960d986cd200) 00010000:00010000:3.0:1785220585.744721:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1785220585.744725:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93d22200/0xf62499ab927287c5 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d369 expref: 487 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.744742:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x948:0x0].0x0 (ffff960d8be41000) 00010000:00010000:3.0:1785220585.744745:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.744749:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90e1d200/0xf62499ab926e71d1 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x948:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b43c51 expref: 486 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220585.745407:0:13876:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510811904 00000100:00100000:0.0:1785220585.745417:0:13876:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510811904 00010000:00010000:3.0:1785220585.745544:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x514:0x0].0x0 (ffff960d89ff2800) 00010000:00010000:3.0:1785220585.745548:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.745555:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8db6d800/0xf62499ab92585b21 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x514:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1167 expref: 485 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745579:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9e6:0x0].0x0 (ffff960d8e5b9400) 00010000:00010000:3.0:1785220585.745582:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.745586:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9a4e8200/0xf62499ab92728e40 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9e6:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d656 expref: 484 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745608:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa5f:0x0].0x0 (ffff960d9013d900) 00010000:00010000:3.0:1785220585.745611:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745615:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9a517000/0xf62499ab9275a976 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa5f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6ebee expref: 483 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745631:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa42:0x0].0x0 (ffff960d8f6ba600) 00010000:00010000:3.0:1785220585.745633:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745637:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95758e00/0xf62499ab927500ab lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa42:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6a16b expref: 482 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745651:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x48e:0x0].0x0 (ffff960d8c221600) 00010000:00010000:3.0:1785220585.745654:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745658:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c19f000/0xf62499ab9255dcae lrc: 0/0,0 mode: --/CR res: [0x240000403:0x48e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa45a3 expref: 481 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745670:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x77e:0x0].0x0 (ffff960d8b340d00) 00010000:00010000:3.0:1785220585.745672:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.745675:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95788e00/0xf62499ab92617a3b lrc: 0/0,0 mode: --/PR res: [0x240000402:0x77e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af3983 expref: 480 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745687:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7a7:0x0].0x0 (ffff960d901f1a00) 00010000:00010000:3.0:1785220585.745699:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745703:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93cd6800/0xf62499ab926695b3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7a7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b0f8aa expref: 479 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745719:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaf9:0x0].0x0 (ffff960d8d85de00) 00010000:00010000:3.0:1785220585.745722:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745725:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d937db400/0xf62499ab9278ce30 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaf9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7df12 expref: 478 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745740:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1c9:0x0].0x0 (ffff960e8aaf6600) 00010000:00010000:3.0:1785220585.745743:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745747:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d884ab600/0xf62499ab924d93ca lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1c9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a765d1 expref: 477 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745773:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3b9:0x0].0x0 (ffff960d8c356500) 00010000:00010000:3.0:1785220585.745776:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745780:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87f06c00/0xf62499ab92536c26 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3b9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a97a1e expref: 476 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745793:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x255:0x0].0x0 (ffff960d89fe5b00) 00010000:00010000:3.0:1785220585.745795:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745798:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d86a4f600/0xf62499ab924f41e1 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x255:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7f93a expref: 475 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745812:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa9d:0x0].0x0 (ffff960d8dd54e00) 00010000:00010000:3.0:1785220585.745815:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.745818:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98c26a00/0xf62499ab92785b90 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xa9d:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a7a5 expref: 474 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745830:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x560:0x0].0x0 (ffff960e86337b00) 00010000:00010000:3.0:1785220585.745832:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745835:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d890bca00/0xf62499ab9257f4f9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x560:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aaef40 expref: 473 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745854:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x35a:0x0].0x0 (ffff960d810af000) 00010000:00010000:3.0:1785220585.745857:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745860:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e3000/0xf62499ab92541bab lrc: 0/0,0 mode: --/CR res: [0x240000402:0x35a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9b4c5 expref: 472 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745878:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8c7:0x0].0x0 (ffff960d95432a00) 00010000:00010000:3.0:1785220585.745880:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745883:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d954c3600/0xf62499ab926b1293 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8c7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b2f5a4 expref: 471 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745899:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x45:0x0].0x0 (ffff960d8a490700) 00010000:00010000:3.0:1785220585.745902:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.745905:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d91273c00/0xf62499ab92789d7d lrc: 0/0,0 mode: --/PR res: [0x240000401:0x45:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c83d expref: 470 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745916:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3a4:0x0].0x0 (ffff960d8bde1b00) 00010000:00010000:3.0:1785220585.745918:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745922:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cd600/0xf62499ab92533f47 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3a4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a96a81 expref: 469 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745938:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x57d:0x0].0x0 (ffff960ea4cd7f00) 00010000:00010000:3.0:1785220585.745940:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.745944:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e241e00/0xf62499ab9259e2e9 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x57d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab8557 expref: 468 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745956:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa50:0x0].0x0 (ffff960d94a78200) 00010000:00010000:3.0:1785220585.745958:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745962:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d958fb000/0xf62499ab9275371c lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6ae5a expref: 467 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745977:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3d7:0x0].0x0 (ffff960e82a72b00) 00010000:00010000:3.0:1785220585.745979:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.745983:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c7aa400/0xf62499ab9256c234 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3d7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa8da9 expref: 466 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.745999:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xaad:0x0].0x0 (ffff960d90213a00) 00010000:00010000:3.0:1785220585.746001:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.746005:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a98e800/0xf62499ab9278ab22 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xaad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ce95 expref: 465 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.746017:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5ea:0x0].0x0 (ffff960d901d9a00) 00010000:00010000:3.0:1785220585.746019:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.746022:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f414200/0xf62499ab925aa70c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x5ea:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1abfeb1 expref: 464 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.746033:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5e6:0x0].0x0 (ffff960d8ddc5a00) 00010000:00010000:3.0:1785220585.746035:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.746038:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9155c000/0xf62499ab9258b484 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5e6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab2c25 expref: 463 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.746053:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x47b:0x0].0x0 (ffff960eacc3eb00) 00010000:00010000:3.0:1785220585.746055:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.746058:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a341000/0xf62499ab925778ce lrc: 0/0,0 mode: --/PR res: [0x240000402:0x47b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aac811 expref: 462 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.746073:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x778:0x0].0x0 (ffff960e828e8500) 00010000:00010000:3.0:1785220585.746076:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.746080:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95496800/0xf62499ab92601cea lrc: 0/0,0 mode: --/CR res: [0x240000403:0x778:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aecf95 expref: 461 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220585.746218:0:13876:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12556 00000100:00100000:0.0:1785220585.746232:0:13876:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0001-mdtlov_UUID+7:7470:x1871938510811904:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1785220585.746546:0:13876:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0001-mdtlov_UUID+8:7470:x1871938510811904:12345-0@lo:1000:osp_up0-1.0 Request processed in 318us (2556us total) trans 4295000084 rc 0/0 00000100:00100000:0.0:1785220585.746554:0:13876:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12556 00000100:00100000:2.0:1785220585.746702:0:7470:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add5c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510811904:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1785220585.746911:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000403:0x7e2:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1785220585.747002:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].1241, 440 off891608 00000040:00080000:2.0:1785220585.747144:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].1245, 440 off1070960 00000004:00080000:2.0:1785220585.747149:0:6516:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff960d8a39a600 with version 2430 00000004:00080000:2.0:1785220585.747157:0:7470:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff960ea686e240 version 2430 rpc_version 2430 00000020:00100000:2.0:1785220585.747178:0:7470:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1785220585.747182:0:7470:0:(client.c:807:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff960d921fa680 x1871938510812160/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1144/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1785220585.747197:0:7470:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d921fa680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510812160:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1785220585.747229:0:7470:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220585.747425:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x20c:0x0].0x0 (ffff960e9af8b100) 00010000:00010000:3.0:1785220585.747427:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747432:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c21ba00/0xf62499ab924e7441 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x20c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7b103 expref: 460 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747451:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x756:0x0].0x0 (ffff960d9571de00) 00010000:00010000:3.0:1785220585.747452:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.747455:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ab66000/0xf62499ab925fb3ff lrc: 0/0,0 mode: --/PR res: [0x240000403:0x756:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aeb516 expref: 459 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747463:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x29e:0x0].0x0 (ffff960d86fd2800) 00010000:00010000:3.0:1785220585.747465:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747467:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d5d600/0xf62499ab9250257d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x29e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a86910 expref: 458 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747477:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x212:0x0].0x0 (ffff960d8bde1300) 00010000:00010000:3.0:1785220585.747478:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747480:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc3a600/0xf62499ab924f4648 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x212:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7fa98 expref: 457 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747488:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x326:0x0].0x0 (ffff960e8ab7ef00) 00010000:00010000:3.0:1785220585.747490:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.747493:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc30400/0xf62499ab925484ab lrc: 0/0,0 mode: --/PR res: [0x240000402:0x326:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9d72b expref: 456 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747500:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3d5:0x0].0x0 (ffff960d8c94bc00) 00010000:00010000:3.0:1785220585.747502:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747504:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfb8ca00/0xf62499ab92548cae lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3d5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9db06 expref: 455 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747511:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x99d:0x0].0x0 (ffff960d8b638800) 00010000:00010000:3.0:1785220585.747512:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747514:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97912c00/0xf62499ab9271fc97 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x99d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b597e2 expref: 454 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747521:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x53e:0x0].0x0 (ffff960e87ac1100) 00010000:00010000:3.0:1785220585.747522:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.747524:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88dbda00/0xf62499ab9258cefc lrc: 0/0,0 mode: --/PR res: [0x240000403:0x53e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab3619 expref: 453 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747533:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5ca:0x0].0x0 (ffff960d8a305c00) 00010000:00010000:3.0:1785220585.747535:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747537:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89bff400/0xf62499ab9259bb27 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x5ca:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab78e6 expref: 452 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747546:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x389:0x0].0x0 (ffff960d8c356000) 00010000:00010000:3.0:1785220585.747547:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747549:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88512200/0xf62499ab9252b960 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x389:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a93c1a expref: 451 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747560:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xad2:0x0].0x0 (ffff960d94987a00) 00010000:00010000:3.0:1785220585.747561:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.747564:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90e1ae00/0xf62499ab92787d94 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xad2:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b781 expref: 450 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747573:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x224:0x0].0x0 (ffff960d8a12d100) 00010000:00010000:3.0:1785220585.747574:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747576:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e2a00/0xf62499ab92501ad3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x224:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a865b3 expref: 449 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747587:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8b2:0x0].0x0 (ffff960d8c2d0200) 00010000:00010000:3.0:1785220585.747588:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.747590:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ea68b4200/0xf62499ab92686e64 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x8b2:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b19c2e expref: 448 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747601:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2e2:0x0].0x0 (ffff960e829ca900) 00010000:00010000:3.0:1785220585.747602:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747604:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2c6800/0xf62499ab92522b30 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2e2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a90f81 expref: 447 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747613:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x55:0x0].0x0 (ffff960d9750f400) 00010000:00010000:3.0:1785220585.747615:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.747617:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e068600/0xf62499ab9278a970 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cdb5 expref: 446 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747626:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6a6:0x0].0x0 (ffff960d85e10f00) 00010000:00010000:3.0:1785220585.747629:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747632:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d966b7e00/0xf62499ab925de397 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6a6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1adaade expref: 445 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747648:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x39:0x0].0x0 (ffff960d912d0a00) 00010000:00010000:3.0:1785220585.747650:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.747654:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b06c800/0xf62499ab9270fe52 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x39:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b50f2a expref: 444 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747666:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x956:0x0].0x0 (ffff960d956f8100) 00010000:00010000:3.0:1785220585.747668:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747672:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9360d800/0xf62499ab926ffdeb lrc: 0/0,0 mode: --/CR res: [0x240000402:0x956:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b4d413 expref: 443 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747687:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x420:0x0].0x0 (ffff960d8faca100) 00010000:00010000:3.0:1785220585.747699:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747703:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c7e000/0xf62499ab92545e47 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x420:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9cba8 expref: 442 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747720:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x301:0x0].0x0 (ffff960d87b49800) 00010000:00010000:3.0:1785220585.747722:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747726:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f4bc00/0xf62499ab92512df5 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x301:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8be05 expref: 441 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.747740:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x389:0x0].0x0 (ffff960d86cfb700) 00010000:00010000:3.0:1785220585.747743:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.747746:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3fa00/0xf62499ab9256f94d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x389:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa9e73 expref: 440 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.748963:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x379:0x0].0x0 (ffff960eba8e7800) 00010000:00010000:3.0:1785220585.748969:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.748977:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cc600/0xf62499ab925425d7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x379:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9b8a0 expref: 439 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.748994:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2d8:0x0].0x0 (ffff960d8c1ed100) 00010000:00010000:3.0:1785220585.748997:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.749001:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ea68b5400/0xf62499ab9250d7da lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2d8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8a347 expref: 438 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749014:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x32c:0x0].0x0 (ffff960d87b81e00) 00010000:00010000:3.0:1785220585.749016:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.749020:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e64cc00/0xf62499ab92577f65 lrc: 0/0,0 mode: --/PR res: [0x200000404:0x32c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aaca5d expref: 437 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749041:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x97b:0x0].0x0 (ffff960d9ac07300) 00010000:00010000:3.0:1785220585.749043:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.749047:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93e33600/0xf62499ab92728d59 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x97b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d5f4 expref: 436 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749060:0:10345:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### wasn't waiting ns: mdt-lustre-MDT0001_UUID lock: ffff960d97980600/0xf62499ab927272e1 lrc: 5/0,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ca63 expref: 435 pid: 13860 timeout: 1057 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749069:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9b0:0x0].0x0 (ffff960d934bd300) 00010000:00010000:3.0:1785220585.749072:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.749076:0:10345:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749847:0:10345:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749860:0:10345:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749924:0:13878:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.749932:0:13878:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220585.750058:0:13878:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a5ed880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_008:lustre-MDT0001-mdtlov_UUID:13878:1871938510812288:0@lo:1000:mdt_io00_008.0 00010000:00010000:3.0:1785220585.750944:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8d4:0x0].0x0 (ffff960d8c39d400) 00010000:00010000:3.0:1785220585.750951:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.750959:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b386200/0xf62499ab926b585b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8d4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b3054f expref: 435 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751002:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9cd:0x0].0x0 (ffff960d8f3ad700) 00010000:00010000:3.0:1785220585.751005:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1785220585.751009:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ff4cc00/0xf62499ab92723f1e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5b4ad expref: 434 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751026:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x33e:0x0].0x0 (ffff960e829ca700) 00010000:00010000:3.0:1785220585.751029:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751032:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88f36200/0xf62499ab9251e2b3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x33e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8f86d expref: 433 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751050:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x40e:0x0].0x0 (ffff960d89376900) 00010000:00010000:3.0:1785220585.751052:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751056:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f3cae00/0xf62499ab92563bba lrc: 0/0,0 mode: --/CR res: [0x240000402:0x40e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa62ad expref: 432 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751073:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x659:0x0].0x0 (ffff960d904c3d00) 00010000:00010000:3.0:1785220585.751075:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751078:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e241a00/0xf62499ab9259745c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x659:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab6162 expref: 431 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751132:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x79c:0x0].0x0 (ffff960d93548700) 00010000:00010000:3.0:1785220585.751135:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751139:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d937f7c00/0xf62499ab9260c97b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x79c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af04d2 expref: 430 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751155:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7c9:0x0].0x0 (ffff960d91403200) 00010000:00010000:3.0:1785220585.751157:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751161:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9162c400/0xf62499ab92629da2 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7c9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afb23c expref: 429 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751175:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x80d:0x0].0x0 (ffff960d8cc44900) 00010000:00010000:3.0:1785220585.751177:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751181:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d923e1600/0xf62499ab92636ddb lrc: 0/0,0 mode: --/CR res: [0x240000403:0x80d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afed53 expref: 428 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751196:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x29c:0x0].0x0 (ffff960e861af900) 00010000:00010000:3.0:1785220585.751199:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751203:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c01b000/0xf62499ab92500a80 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x29c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a86049 expref: 427 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751217:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xa34:0x0].0x0 (ffff960d8d770a00) 00010000:00010000:3.0:1785220585.751220:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751224:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93d67800/0xf62499ab927813a6 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xa34:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7853f expref: 426 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751237:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x212:0x0].0x0 (ffff960e9ac51200) 00010000:00010000:3.0:1785220585.751240:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751244:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bf94a00/0xf62499ab924ea1f9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x212:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7c11e expref: 425 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751256:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x352:0x0].0x0 (ffff960d87ba5e00) 00010000:00010000:3.0:1785220585.751258:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751262:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e4c00/0xf62499ab92520d62 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x352:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9063c expref: 424 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751279:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4c9:0x0].0x0 (ffff960e86f6a800) 00010000:00010000:3.0:1785220585.751282:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751286:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88510000/0xf62499ab9256639f lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4c9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa707c expref: 423 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751302:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaed:0x0].0x0 (ffff960d97ba4100) 00010000:00010000:3.0:1785220585.751305:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751308:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d913e2400/0xf62499ab9278b348 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaed:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d2af expref: 422 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751330:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9c9:0x0].0x0 (ffff960d8f2d7f00) 00010000:00010000:3.0:1785220585.751332:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1785220585.751336:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8aed3400/0xf62499ab92722a1e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ab99 expref: 421 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751352:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x81f:0x0].0x0 (ffff960d8d666c00) 00010000:00010000:3.0:1785220585.751355:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751358:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc3a200/0xf62499ab9263f54a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x81f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b00e2a expref: 420 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751373:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xab2:0x0].0x0 (ffff960e8121fd00) 00010000:00010000:3.0:1785220585.751376:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751380:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d96512600/0xf62499ab92782201 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xab2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b78c46 expref: 419 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751396:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaf7:0x0].0x0 (ffff960d903d3500) 00010000:00010000:3.0:1785220585.751398:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751402:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b026800/0xf62499ab9278ca01 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaf7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dd52 expref: 418 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751417:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2e2:0x0].0x0 (ffff960e829ca900) 00010000:00010000:3.0:1785220585.751419:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1785220585.751423:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c21b600/0xf62499ab92520c97 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x2e2:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9058d expref: 417 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751436:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1d8:0x0].0x0 (ffff960d8d983400) 00010000:00010000:3.0:1785220585.751438:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751442:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c7e800/0xf62499ab924dd5da lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1d8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a77a06 expref: 416 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751459:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x50f:0x0].0x0 (ffff960d8cc44800) 00010000:00010000:3.0:1785220585.751462:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751465:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfaca200/0xf62499ab92576215 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x50f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aac0d9 expref: 415 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751479:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3c9:0x0].0x0 (ffff960d87ba6a00) 00010000:00010000:3.0:1785220585.751481:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751485:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89ece800/0xf62499ab9253ffc7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3c9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9ab80 expref: 414 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751498:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x424:0x0].0x0 (ffff960eba991f00) 00010000:00010000:3.0:1785220585.751501:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.751504:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f820c00/0xf62499ab9257c11a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x424:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aade84 expref: 413 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751518:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x97b:0x0].0x0 (ffff960d9ac07300) 00010000:00010000:3.0:1785220585.751520:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.751524:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93715c00/0xf62499ab9275dc9f lrc: 0/0,0 mode: --/CR res: [0x240000403:0x97b:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6fb99 expref: 412 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751539:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x371:0x0].0x0 (ffff960d87ba4000) 00010000:00010000:3.0:1785220585.751541:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751545:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c7fe00/0xf62499ab92538540 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x371:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a98283 expref: 411 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751562:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3d4:0x0].0x0 (ffff960e9af8bb00) 00010000:00010000:3.0:1785220585.751565:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751568:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc33600/0xf62499ab92539f8e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3d4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a98a78 expref: 410 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751584:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xae5:0x0].0x0 (ffff960d9118dc00) 00010000:00010000:3.0:1785220585.751587:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.751591:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8eecc200/0xf62499ab9278ab5a lrc: 0/0,0 mode: --/PR res: [0x240000403:0xae5:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cea3 expref: 409 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751603:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x6f3:0x0].0x0 (ffff960d94c6d100) 00010000:00010000:3.0:1785220585.751605:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751609:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9351c800/0xf62499ab925d679d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x6f3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad7490 expref: 408 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751633:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9fb:0x0].0x0 (ffff960d8bc4e500) 00010000:00010000:3.0:1785220585.751636:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.751639:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d978eda00/0xf62499ab9273993c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9fb:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b63b66 expref: 407 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751653:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x695:0x0].0x0 (ffff960d8e5e8f00) 00010000:00010000:3.0:1785220585.751656:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.751659:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d1da200/0xf62499ab925aa8b7 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x695:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1abfefe expref: 406 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751673:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x66a:0x0].0x0 (ffff960d901da300) 00010000:00010000:3.0:1785220585.751676:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751681:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92380c00/0xf62499ab9259a3fe lrc: 0/0,0 mode: --/CR res: [0x240000403:0x66a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab71ed expref: 405 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751718:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3b2:0x0].0x0 (ffff960d86cfb900) 00010000:00010000:3.0:1785220585.751721:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751726:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87f07800/0xf62499ab925345c9 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3b2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a96c5d expref: 404 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.751740:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4c7:0x0].0x0 (ffff960d8da0d600) 00010000:00010000:3.0:1785220585.751743:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.751747:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8baf2e00/0xf62499ab925657c8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4c7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa6b12 expref: 403 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752072:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2df:0x0].0x0 (ffff960d89d21f00) 00010000:00010000:3.0:1785220585.752075:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752080:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89ac0800/0xf62499ab9250db61 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2df:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8a474 expref: 402 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752170:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3a2:0x0].0x0 (ffff960d892b9200) 00010000:00010000:3.0:1785220585.752172:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752176:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a38d000/0xf62499ab92544033 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3a2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9c095 expref: 401 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752188:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x427:0x0].0x0 (ffff960d87b81900) 00010000:00010000:3.0:1785220585.752191:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752193:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e2400/0xf62499ab92548a70 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x427:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9da65 expref: 400 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752213:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x818:0x0].0x0 (ffff960e8cfebc00) 00010000:00010000:3.0:1785220585.752216:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752219:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90e1b400/0xf62499ab9263b7c4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x818:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b00524 expref: 399 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752233:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x38f:0x0].0x0 (ffff960d8facae00) 00010000:00010000:3.0:1785220585.752236:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752239:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a277e00/0xf62499ab9254311b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x38f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9bcf9 expref: 398 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752251:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9db:0x0].0x0 (ffff960d975d6700) 00010000:00010000:3.0:1785220585.752253:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.752267:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7fc00/0xf62499ab9272864b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9db:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d297 expref: 397 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752283:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x49d:0x0].0x0 (ffff960d8cc44b00) 00010000:00010000:3.0:1785220585.752285:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752289:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e284a00/0xf62499ab9255dc76 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x49d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa4580 expref: 396 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752303:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7e3:0x0].0x0 (ffff960d8ba66c00) 00010000:00010000:3.0:1785220585.752305:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752309:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d96494c00/0xf62499ab92621fef lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7e3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af68e6 expref: 395 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752325:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x97d:0x0].0x0 (ffff960d9012a900) 00010000:00010000:3.0:1785220585.752328:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752332:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d978d6c00/0xf62499ab92720b38 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x97d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b59e48 expref: 394 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752344:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1f3:0x0].0x0 (ffff960e930c5800) 00010000:00010000:3.0:1785220585.752347:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752351:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88fd8c00/0xf62499ab924f87cc lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1f3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a80f7c expref: 393 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752367:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x2c4:0x0].0x0 (ffff960e930c5600) 00010000:00010000:3.0:1785220585.752370:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.752373:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b1e1000/0xf62499ab92567f83 lrc: 0/0,0 mode: --/PR res: [0x200000404:0x2c4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa7832 expref: 392 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752391:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x603:0x0].0x0 (ffff960d8faca900) 00010000:00010000:3.0:1785220585.752393:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752397:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f3ab400/0xf62499ab9258cca2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x603:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab3508 expref: 391 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752413:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9fa:0x0].0x0 (ffff960d8a1bcd00) 00010000:00010000:3.0:1785220585.752415:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752419:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c1b000/0xf62499ab927328a2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9fa:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b61b3e expref: 390 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752434:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x306:0x0].0x0 (ffff960d8bde2900) 00010000:00010000:3.0:1785220585.752437:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752440:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6ca000/0xf62499ab92514e40 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x306:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8c7c8 expref: 389 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752460:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x552:0x0].0x0 (ffff960d8da0d200) 00010000:00010000:3.0:1785220585.752463:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.752467:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d892b6800/0xf62499ab92588624 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x552:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1e64 expref: 388 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752481:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x2f:0x0].0x0 (ffff960d9a4b7c00) 00010000:00010000:3.0:1785220585.752483:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.752487:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b06ec00/0xf62499ab9268668b lrc: 0/0,0 mode: --/PR res: [0x240000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b19a2f expref: 387 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752502:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x680:0x0].0x0 (ffff960d89bf5a00) 00010000:00010000:3.0:1785220585.752504:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.752508:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d937a9c00/0xf62499ab9259e313 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x680:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab8565 expref: 386 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752521:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa94:0x0].0x0 (ffff960d8d833b00) 00010000:00010000:3.0:1785220585.752523:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752526:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ab60600/0xf62499ab9277a042 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76b3e expref: 385 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752543:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x39:0x0].0x0 (ffff960d912d0a00) 00010000:00010000:3.0:1785220585.752545:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.752549:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8db7ee00/0xf62499ab9270f878 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b50cec expref: 384 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752563:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x36a:0x0].0x0 (ffff960ead0e5b00) 00010000:00010000:3.0:1785220585.752565:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752570:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e284600/0xf62499ab92525c84 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x36a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a91f9c expref: 383 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752587:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x10:0x0].0x0 (ffff960d8c69f000) 00010000:00010000:3.0:1785220585.752589:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.752593:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cd200/0xf62499ab925464fa lrc: 0/0,0 mode: --/PR res: [0x240000402:0x10:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9cd84 expref: 382 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752611:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4f3:0x0].0x0 (ffff960d8d983100) 00010000:00010000:3.0:1785220585.752613:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.752617:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8db6de00/0xf62499ab9256ada4 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4f3:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa867f expref: 381 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752633:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8cd:0x0].0x0 (ffff960d87fce900) 00010000:00010000:3.0:1785220585.752635:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752639:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94975000/0xf62499ab926a0c13 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x8cd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b26c8a expref: 380 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752656:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x420:0x0].0x0 (ffff960d89d21900) 00010000:00010000:3.0:1785220585.752659:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752663:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d67400/0xf62499ab92547c62 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x420:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9d48b expref: 379 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752678:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7dc:0x0].0x0 (ffff960d8ab99b00) 00010000:00010000:3.0:1785220585.752680:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752684:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9162d200/0xf62499ab9262251a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7dc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af6a83 expref: 378 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752717:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x31:0x0].0x0 (ffff960d8b7ff200) 00010000:00010000:3.0:1785220585.752719:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.752723:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90ef3800/0xf62499ab926995bb lrc: 0/0,0 mode: --/PR res: [0x240000401:0x31:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b2294d expref: 377 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752738:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x348:0x0].0x0 (ffff960d8bde2f00) 00010000:00010000:3.0:1785220585.752740:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752744:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bf94800/0xf62499ab9252b98a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x348:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a93c28 expref: 376 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752756:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x50b:0x0].0x0 (ffff960d9061c600) 00010000:00010000:3.0:1785220585.752759:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752855:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f292a00/0xf62499ab92581616 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x50b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aafb4f expref: 375 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752875:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6bf:0x0].0x0 (ffff960d8eaf8600) 00010000:00010000:3.0:1785220585.752877:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752881:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2b0a00/0xf62499ab925e0015 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6bf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1adb63e expref: 374 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752907:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9da:0x0].0x0 (ffff960d98544f00) 00010000:00010000:3.0:1785220585.752910:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.752914:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97552c00/0xf62499ab9273cdb5 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x9da:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b649b3 expref: 373 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752927:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x2b9:0x0].0x0 (ffff960e82a72900) 00010000:00010000:3.0:1785220585.752929:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.752932:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88dbca00/0xf62499ab92582202 lrc: 0/0,0 mode: --/PR res: [0x200000404:0x2b9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aaff69 expref: 372 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752947:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x917:0x0].0x0 (ffff960d91330100) 00010000:00010000:3.0:1785220585.752950:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.752953:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9643e200/0xf62499ab926c8ff7 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x917:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b357c7 expref: 371 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752965:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1d1:0x0].0x0 (ffff960ec20ff400) 00010000:00010000:3.0:1785220585.752967:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752971:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3ee00/0xf62499ab924d9e89 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1d1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a76a77 expref: 370 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.752983:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x903:0x0].0x0 (ffff960d98544300) 00010000:00010000:3.0:1785220585.752985:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.752989:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9767e000/0xf62499ab926c4210 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x903:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b344bf expref: 369 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753000:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2b5:0x0].0x0 (ffff960e9ac51c00) 00010000:00010000:3.0:1785220585.753003:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753006:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d5c600/0xf62499ab92506509 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2b5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a87ee2 expref: 368 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753020:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6a4:0x0].0x0 (ffff960d8c3e2c00) 00010000:00010000:3.0:1785220585.753023:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1785220585.753026:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9230a000/0xf62499ab9268e2c4 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6a4:0x0].0x0 bits 0x1a/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b1bd13 expref: 367 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753045:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x827:0x0].0x0 (ffff960d956be600) 00010000:00010000:3.0:1785220585.753048:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753052:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b374800/0xf62499ab92645fe0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x827:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b02220 expref: 366 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753064:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x915:0x0].0x0 (ffff960d96538200) 00010000:00010000:3.0:1785220585.753066:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753069:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f408a00/0xf62499ab926c7c16 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x915:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b3533d expref: 365 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753391:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x78c:0x0].0x0 (ffff960d90a16b00) 00010000:00010000:3.0:1785220585.753395:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753401:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95691400/0xf62499ab9260a134 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x78c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aef711 expref: 364 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753424:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7e2:0x0].0x0 (ffff960d8b539500) 00010000:00010000:3.0:1785220585.753427:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.753430:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92287a00/0xf62499ab9263571b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7e2:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afe748 expref: 363 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753446:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa72:0x0].0x0 (ffff960d9498e600) 00010000:00010000:3.0:1785220585.753449:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753452:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b0ebe00/0xf62499ab92763ced lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b708c7 expref: 362 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753468:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3f1:0x0].0x0 (ffff960d8a12ce00) 00010000:00010000:3.0:1785220585.753471:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753475:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bf95a00/0xf62499ab9253e445 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3f1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9a2f8 expref: 361 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753494:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3db:0x0].0x0 (ffff960ea686f500) 00010000:00010000:3.0:1785220585.753497:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753500:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e3a00/0xf62499ab92550a8b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3db:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa0243 expref: 360 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753515:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xae8:0x0].0x0 (ffff960d98bb1800) 00010000:00010000:3.0:1785220585.753518:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753522:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a98e400/0xf62499ab9278ae4e lrc: 0/0,0 mode: --/CR res: [0x240000403:0xae8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d040 expref: 359 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753537:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3b7:0x0].0x0 (ffff960eacc3ea00) 00010000:00010000:3.0:1785220585.753540:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753543:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8dc11000/0xf62499ab9253fa9c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3b7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9a9c0 expref: 358 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753559:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa55:0x0].0x0 (ffff960d90a77e00) 00010000:00010000:3.0:1785220585.753561:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753564:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a22fc00/0xf62499ab92756ef9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa55:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6d892 expref: 357 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753579:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x58f:0x0].0x0 (ffff960d8ab88f00) 00010000:00010000:3.0:1785220585.753581:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753585:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d0c200/0xf62499ab9258fdbe lrc: 0/0,0 mode: --/CR res: [0x240000403:0x58f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab43c5 expref: 356 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753604:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x91e:0x0].0x0 (ffff960d8ad57600) 00010000:00010000:3.0:1785220585.753606:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753609:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d914d2000/0xf62499ab926ca7d6 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x91e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b35d31 expref: 355 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753625:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa9d:0x0].0x0 (ffff960d8dd54e00) 00010000:00010000:3.0:1785220585.753628:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.753632:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c21a800/0xf62499ab9277fc76 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xa9d:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77aaa expref: 354 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753643:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x674:0x0].0x0 (ffff960d901d3f00) 00010000:00010000:3.0:1785220585.753645:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753649:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d903c8e00/0xf62499ab9259c297 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x674:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab7b40 expref: 353 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753664:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x951:0x0].0x0 (ffff960d9908ef00) 00010000:00010000:3.0:1785220585.753667:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753670:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9ae0e200/0xf62499ab926eac8d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x951:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b449cc expref: 352 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753685:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x98a:0x0].0x0 (ffff960d959a3600) 00010000:00010000:3.0:1785220585.753687:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753702:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d965d9200/0xf62499ab92710330 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x98a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b51106 expref: 351 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753720:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xae5:0x0].0x0 (ffff960d9118dc00) 00010000:00010000:3.0:1785220585.753722:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.753727:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d959c1800/0xf62499ab9278cbf9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xae5:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7de4e expref: 350 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.753741:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e0:0x0].0x0 (ffff960eacc3e500) 00010000:00010000:3.0:1785220585.753744:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.753747:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c10e00/0xf62499ab924df31c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a782b8 expref: 349 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.755862:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x738:0x0].0x0 (ffff960d8f432e00) 00010000:00010000:3.0:1785220585.755873:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.755885:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95a22e00/0xf62499ab925f1c28 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x738:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae6584 expref: 348 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.755918:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3d4:0x0].0x0 (ffff960eba97e800) 00010000:00010000:3.0:1785220585.755921:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.755926:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d906e6600/0xf62499ab92542885 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3d4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9bafa expref: 347 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.755940:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3e4:0x0].0x0 (ffff960d8da0d500) 00010000:00010000:3.0:1785220585.755943:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.755947:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8dc10c00/0xf62499ab9253c9b8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3e4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a999b3 expref: 346 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.755965:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3f4:0x0].0x0 (ffff960eba97e300) 00010000:00010000:3.0:1785220585.755968:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.755971:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8971c200/0xf62499ab9254711e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3f4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9d0ef expref: 345 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.755992:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9e1:0x0].0x0 (ffff960d8eb29f00) 00010000:00010000:3.0:1785220585.755994:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.755997:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c08d400/0xf62499ab927281ac lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9e1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d098 expref: 344 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756013:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x813:0x0].0x0 (ffff960d959b7800) 00010000:00010000:3.0:1785220585.756016:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756020:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d937b1e00/0xf62499ab926383a6 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x813:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aff437 expref: 343 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756036:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x816:0x0].0x0 (ffff960d90deb700) 00010000:00010000:3.0:1785220585.756039:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756042:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d902ea000/0xf62499ab9263b038 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x816:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b002b5 expref: 342 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756061:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x480:0x0].0x0 (ffff960d8c69fa00) 00010000:00010000:3.0:1785220585.756063:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756067:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a039400/0xf62499ab925882ce lrc: 0/0,0 mode: --/CR res: [0x240000402:0x480:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1d14 expref: 341 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756099:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3e1:0x0].0x0 (ffff960d86fd2900) 00010000:00010000:3.0:1785220585.756163:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756169:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc33c00/0xf62499ab925461ff lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3e1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9cca4 expref: 340 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756190:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x437:0x0].0x0 (ffff960d8aee6200) 00010000:00010000:3.0:1785220585.756193:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756197:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c018200/0xf62499ab9254b0c6 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x437:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9e674 expref: 339 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756212:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4f5:0x0].0x0 (ffff960d906e2100) 00010000:00010000:3.0:1785220585.756215:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756219:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f821600/0xf62499ab9256e08e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4f5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa97b9 expref: 338 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756235:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8de:0x0].0x0 (ffff960d8fdb2100) 00010000:00010000:3.0:1785220585.756237:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756241:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc85400/0xf62499ab926e01ca lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8de:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b41f40 expref: 337 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756255:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4e1:0x0].0x0 (ffff960d87ba2200) 00010000:00010000:3.0:1785220585.756258:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756262:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d86a4ee00/0xf62499ab92569188 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4e1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa7e28 expref: 336 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756278:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x322:0x0].0x0 (ffff960e8cfeb200) 00010000:00010000:3.0:1785220585.756281:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.756284:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc32000/0xf62499ab9251c3d4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x322:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8eec6 expref: 335 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756300:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x525:0x0].0x0 (ffff960d8bde1000) 00010000:00010000:3.0:1785220585.756302:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756305:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f9ed000/0xf62499ab9257a79e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x525:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aad5fc expref: 334 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756327:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x710:0x0].0x0 (ffff960d89a82d00) 00010000:00010000:3.0:1785220585.756330:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756334:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93d32000/0xf62499ab925f5b4b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x710:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae796c expref: 333 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756349:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2f2:0x0].0x0 (ffff960eacc3be00) 00010000:00010000:3.0:1785220585.756352:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756356:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc32600/0xf62499ab925114e2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2f2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8b6db expref: 332 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756369:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3e8:0x0].0x0 (ffff960d8c221800) 00010000:00010000:3.0:1785220585.756372:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756376:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e5e00/0xf62499ab9253e556 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3e8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9a337 expref: 331 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756389:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4d4:0x0].0x0 (ffff960e86f6a600) 00010000:00010000:3.0:1785220585.756392:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756395:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8971c400/0xf62499ab9255ffa7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4d4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa50a1 expref: 330 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756410:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2ff:0x0].0x0 (ffff960d892b9700) 00010000:00010000:3.0:1785220585.756413:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756417:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87f07c00/0xf62499ab925139a2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2ff:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8c13f expref: 329 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756430:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7dc:0x0].0x0 (ffff960d8ab99b00) 00010000:00010000:3.0:1785220585.756433:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.756437:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8dd47c00/0xf62499ab9262234c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x7dc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af69e2 expref: 328 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756452:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x7c5:0x0].0x0 (ffff960d975d6300) 00010000:00010000:3.0:1785220585.756454:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.756458:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93452e00/0xf62499ab927285f0 lrc: 0/0,0 mode: --/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d266 expref: 327 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756474:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9fa:0x0].0x0 (ffff960d906c9200) 00010000:00010000:3.0:1785220585.756477:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.756480:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea2c000/0xf62499ab9275251e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x9fa:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6a90c expref: 326 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756494:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaa0:0x0].0x0 (ffff960d898d0a00) 00010000:00010000:3.0:1785220585.756497:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756501:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95956800/0xf62499ab9277ff0f lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77bd7 expref: 325 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756514:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x386:0x0].0x0 (ffff960d810afc00) 00010000:00010000:3.0:1785220585.756517:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.756521:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9c1800/0xf62499ab9258aec6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x386:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab2a73 expref: 324 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756533:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x324:0x0].0x0 (ffff960ead0e5a00) 00010000:00010000:3.0:1785220585.756535:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756539:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89159200/0xf62499ab9251aa12 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x324:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8e5f1 expref: 323 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756562:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2fa:0x0].0x0 (ffff960d8c94b500) 00010000:00010000:3.0:1785220585.756565:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.756569:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ddaea00/0xf62499ab92515929 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x2fa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8cc60 expref: 322 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756581:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x76a:0x0].0x0 (ffff960d95661200) 00010000:00010000:3.0:1785220585.756584:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756587:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d959ecc00/0xf62499ab925fc379 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x76a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aebc08 expref: 321 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756604:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa1a:0x0].0x0 (ffff960d90798700) 00010000:00010000:3.0:1785220585.756606:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.756610:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94ac7800/0xf62499ab92741019 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xa1a:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b65b6b expref: 320 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756625:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x63f:0x0].0x0 (ffff960d8cfff300) 00010000:00010000:3.0:1785220585.756628:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756632:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f3aa800/0xf62499ab92591009 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x63f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab485d expref: 319 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756643:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x903:0x0].0x0 (ffff960d98544300) 00010000:00010000:3.0:1785220585.756645:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.756649:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b040200/0xf62499ab926c3f54 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x903:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b3441e expref: 318 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756664:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x464:0x0].0x0 (ffff960d8bde1200) 00010000:00010000:3.0:1785220585.756667:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.756670:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d906e7600/0xf62499ab92565eba lrc: 0/0,0 mode: --/CR res: [0x240000403:0x464:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa6e1b expref: 317 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756684:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xad6:0x0].0x0 (ffff960d9646c400) 00010000:00010000:3.0:1785220585.756687:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756775:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90e1a200/0xf62499ab92788264 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xad6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ba59 expref: 316 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756795:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6cb:0x0].0x0 (ffff960d8e02fa00) 00010000:00010000:3.0:1785220585.756797:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.756801:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8eecd400/0xf62499ab925e3d32 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6cb:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1adcae3 expref: 315 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756816:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1a9:0x0].0x0 (ffff960d87ba3200) 00010000:00010000:3.0:1785220585.756819:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756822:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9fe800/0xf62499ab925111d9 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1a9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8b5bc expref: 314 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756838:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xad8:0x0].0x0 (ffff960d8c76b700) 00010000:00010000:3.0:1785220585.756840:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756844:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89ab6200/0xf62499ab92788749 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xad8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7bd46 expref: 313 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756858:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3e6:0x0].0x0 (ffff960d8c1ed400) 00010000:00010000:3.0:1785220585.756860:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756864:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c7ec00/0xf62499ab9253cc9e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3e6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a99aa1 expref: 312 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756883:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x20:0x0].0x0 (ffff960d9233d600) 00010000:00010000:3.0:1785220585.756886:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.756890:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93f52a00/0xf62499ab925e095a lrc: 0/0,0 mode: --/PR res: [0x240000401:0x20:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1adb81a expref: 311 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756902:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7af:0x0].0x0 (ffff960d90128600) 00010000:00010000:3.0:1785220585.756904:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756908:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8dcbbe00/0xf62499ab926128f7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7af:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af2635 expref: 310 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756923:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa20:0x0].0x0 (ffff960d9ae65b00) 00010000:00010000:3.0:1785220585.756926:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.756930:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94b40000/0xf62499ab92763824 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa20:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b70865 expref: 309 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756943:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa6a:0x0].0x0 (ffff960d8e6a7100) 00010000:00010000:3.0:1785220585.756946:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756949:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d948a4a00/0xf62499ab927606de lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b700af expref: 308 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756965:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffff960d8bc2b500) 00010000:00010000:3.0:1785220585.756967:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.756971:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cc200/0xf62499ab924ed30e lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7d067 expref: 307 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.756983:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x69c:0x0].0x0 (ffff960e8cd4f400) 00010000:00010000:3.0:1785220585.756986:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.756990:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9036ba00/0xf62499ab925abdd3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x69c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ac0396 expref: 306 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757005:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9c2:0x0].0x0 (ffff960d94d52c00) 00010000:00010000:3.0:1785220585.757007:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757011:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90d79200/0xf62499ab92721ac7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9c2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5a4ed expref: 305 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757026:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8bd:0x0].0x0 (ffff960d92386a00) 00010000:00010000:3.0:1785220585.757029:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757033:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d923ea200/0xf62499ab926aa3b9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8bd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b2ccd1 expref: 304 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757055:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9e8:0x0].0x0 (ffff960e8518c800) 00010000:00010000:3.0:1785220585.757058:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757061:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94fcaa00/0xf62499ab9275a460 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x9e8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6ea4a expref: 303 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757074:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5:0x0].0x0 (ffff960ead0e0900) 00010000:00010000:3.0:1785220585.757077:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.757080:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c21b400/0xf62499ab924f1430 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7e988 expref: 302 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757123:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x753:0x0].0x0 (ffff960d893d0300) 00010000:00010000:3.0:1785220585.757127:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757131:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d922da400/0xf62499ab925f5f50 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x753:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae7b09 expref: 301 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757152:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9c8:0x0].0x0 (ffff960d90d33300) 00010000:00010000:3.0:1785220585.757155:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757159:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d935d7c00/0xf62499ab92722b98 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9c8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5abf4 expref: 300 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757174:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x57d:0x0].0x0 (ffff960ea4cd7f00) 00010000:00010000:3.0:1785220585.757176:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757180:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d900e5800/0xf62499ab92585333 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x57d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab0f14 expref: 299 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757195:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6a3:0x0].0x0 (ffff960ebfb70900) 00010000:00010000:3.0:1785220585.757197:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757201:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95497800/0xf62499ab925e2b26 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6a3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1adc24d expref: 298 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757215:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x74e:0x0].0x0 (ffff960d8c3e3700) 00010000:00010000:3.0:1785220585.757218:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757221:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9219ba00/0xf62499ab925f54f3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x74e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae776d expref: 297 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757234:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5a5:0x0].0x0 (ffff960d8c1ec100) 00010000:00010000:3.0:1785220585.757236:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757240:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8acc8a00/0xf62499ab92587592 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5a5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab19b7 expref: 296 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757254:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9c7:0x0].0x0 (ffff960d8ede5e00) 00010000:00010000:3.0:1785220585.757256:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.757260:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9495fa00/0xf62499ab9272296f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9c7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ab45 expref: 295 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757275:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x32c:0x0].0x0 (ffff960d87ba6300) 00010000:00010000:3.0:1785220585.757277:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757281:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89a5a800/0xf62499ab9251b7cc lrc: 0/0,0 mode: --/CR res: [0x240000403:0x32c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8eb2a expref: 294 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757297:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x38e:0x0].0x0 (ffff960d8bde2b00) 00010000:00010000:3.0:1785220585.757299:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757304:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c01be00/0xf62499ab9252d4fe lrc: 0/0,0 mode: --/CR res: [0x240000403:0x38e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a943ec expref: 293 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757317:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xc:0x0].0x0 (ffff960eba97e100) 00010000:00010000:3.0:1785220585.757320:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757323:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d892b7e00/0xf62499ab924f19af lrc: 0/0,0 mode: --/CR res: [0x240000402:0xc:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7ebc6 expref: 292 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757335:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x10f:0x0].0x0 (ffff960e839f7600) 00010000:00010000:3.0:1785220585.757337:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.757341:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d892b6000/0xf62499ab924d7ada lrc: 0/0,0 mode: --/PR res: [0x240000402:0x10f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75ef4 expref: 291 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757354:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x354:0x0].0x0 (ffff960d87b81c00) 00010000:00010000:3.0:1785220585.757356:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757360:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d892b7200/0xf62499ab92521278 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x354:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a90826 expref: 290 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757381:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3ae:0x0].0x0 (ffff960e8cfeb900) 00010000:00010000:3.0:1785220585.757383:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757387:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ce61200/0xf62499ab92544b7e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3ae:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9c4fc expref: 289 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757403:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xaa4:0x0].0x0 (ffff960d8cd4b400) 00010000:00010000:3.0:1785220585.757406:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757409:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88db4400/0xf62499ab927899d3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xaa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7c65a expref: 288 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757420:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x52b:0x0].0x0 (ffff960e839f7900) 00010000:00010000:3.0:1785220585.757423:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757427:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cd400/0xf62499ab9257bd15 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x52b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aadd42 expref: 287 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757443:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5af:0x0].0x0 (ffff960d8c220c00) 00010000:00010000:3.0:1785220585.757445:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757449:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cf31a00/0xf62499ab92587ec9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5af:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1c34 expref: 286 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757463:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1d9:0x0].0x0 (ffff960d86fd2a00) 00010000:00010000:3.0:1785220585.757465:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757469:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f3da00/0xf62499ab924dd2a7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1d9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a77903 expref: 285 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757483:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x45b:0x0].0x0 (ffff960d87ba2e00) 00010000:00010000:3.0:1785220585.757486:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757490:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c5de600/0xf62499ab92550d4e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x45b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa0300 expref: 284 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757505:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x510:0x0].0x0 (ffff960d8a12d700) 00010000:00010000:3.0:1785220585.757508:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757512:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88f36800/0xf62499ab9257605c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x510:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aabfd6 expref: 283 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757526:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x76e:0x0].0x0 (ffff960d97b95100) 00010000:00010000:3.0:1785220585.757528:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.757532:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95612400/0xf62499ab925ffb41 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x76e:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aec75a expref: 282 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757548:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.757551:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.757554:0:10345:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 4 ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757564:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ea68b4000/0xf62499ab92488422 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x10/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a5a2f9 expref: 281 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757576:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x6ab:0x0].0x0 (ffff960d8bc4e600) 00010000:00010000:3.0:1785220585.757578:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757583:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9059a600/0xf62499ab925b79b4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x6ab:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ac7efd expref: 280 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757599:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x26e:0x0].0x0 (ffff960d8a4a3a00) 00010000:00010000:3.0:1785220585.757601:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757605:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3e200/0xf62499ab92520478 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x26e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9041a expref: 279 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757618:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x31d:0x0].0x0 (ffff960e86f6aa00) 00010000:00010000:3.0:1785220585.757620:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.757624:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f4b400/0xf62499ab925147cc lrc: 0/0,0 mode: --/PR res: [0x240000402:0x31d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8c567 expref: 278 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757635:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3e4:0x0].0x0 (ffff960d8bde2c00) 00010000:00010000:3.0:1785220585.757638:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757641:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c72e00/0xf62499ab925406f1 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3e4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9ade8 expref: 277 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757656:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x687:0x0].0x0 (ffff960d8f98ff00) 00010000:00010000:3.0:1785220585.757658:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757662:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d906e7200/0xf62499ab925d55c9 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x687:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad6ec4 expref: 276 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757676:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x69e:0x0].0x0 (ffff960d94faac00) 00010000:00010000:3.0:1785220585.757678:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757682:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94b71e00/0xf62499ab925b256e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x69e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ac64fc expref: 275 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757709:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x876:0x0].0x0 (ffff960d98a28b00) 00010000:00010000:3.0:1785220585.757712:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757716:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b040600/0xf62499ab92689e8b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x876:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b1a9e8 expref: 274 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757730:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1be:0x0].0x0 (ffff960d8ce3ee00) 00010000:00010000:3.0:1785220585.757733:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.757736:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a274a00/0xf62499ab9251e1e8 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1be:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8f819 expref: 273 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757749:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaef:0x0].0x0 (ffff960d8a2bfe00) 00010000:00010000:3.0:1785220585.757752:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757755:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93687c00/0xf62499ab9278b5be lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaef:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d3f8 expref: 272 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757769:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2f4:0x0].0x0 (ffff960d8c218400) 00010000:00010000:3.0:1785220585.757772:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757775:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a38cc00/0xf62499ab92511dbe lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2f4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8b966 expref: 271 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757792:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x46:0x0].0x0 (ffff960d8ac26a00) 00010000:00010000:3.0:1785220585.757794:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.757798:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a020a00/0xf62499ab926fd2fd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b4c74e expref: 270 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757812:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3c3:0x0].0x0 (ffff960d8c218800) 00010000:00010000:3.0:1785220585.757815:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.757818:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc3b800/0xf62499ab92540291 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3c3:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9ac8a expref: 269 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757830:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2e2:0x0].0x0 (ffff960e829ca900) 00010000:00010000:3.0:1785220585.757832:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.757836:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebdb2b200/0xf62499ab92522fd6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x2e2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9113a expref: 268 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757853:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5fe:0x0].0x0 (ffff960d8aee6800) 00010000:00010000:3.0:1785220585.757856:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757860:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8da8c800/0xf62499ab9258cabf lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5fe:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab3475 expref: 267 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757877:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7cc:0x0].0x0 (ffff960d8d666e00) 00010000:00010000:3.0:1785220585.757879:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757883:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93716200/0xf62499ab92629d32 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7cc:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afb227 expref: 266 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757897:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x26b:0x0].0x0 (ffff960e930c5500) 00010000:00010000:3.0:1785220585.757899:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757903:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89ecea00/0xf62499ab924f705d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x26b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a80772 expref: 265 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757919:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffff960d8bc2b500) 00010000:00010000:3.0:1785220585.757921:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757925:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a277600/0xf62499ab924ea078 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x15b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7c0ca expref: 264 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757940:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x739:0x0].0x0 (ffff960d8b2ffe00) 00010000:00010000:3.0:1785220585.757943:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.757947:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ba7bc00/0xf62499ab925ef86b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x739:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae5665 expref: 263 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757961:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x6f9:0x0].0x0 (ffff960d957f8700) 00010000:00010000:3.0:1785220585.757964:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.757967:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8faeb400/0xf62499ab925d8c33 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x6f9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad83d9 expref: 262 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757979:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x11:0x0].0x0 (ffff960d8bde2800) 00010000:00010000:3.0:1785220585.757982:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.757986:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f4aa00/0xf62499ab9250071c lrc: 0/0,0 mode: --/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a85f46 expref: 261 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.757999:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1c3:0x0].0x0 (ffff960eba97e000) 00010000:00010000:3.0:1785220585.758002:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758006:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c01b200/0xf62499ab924f2793 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1c3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7f0ce expref: 260 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758019:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x756:0x0].0x0 (ffff960d9571de00) 00010000:00010000:3.0:1785220585.758022:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758025:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d909d1e00/0xf62499ab925fb5a3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x756:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aeb5ef expref: 259 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758039:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1a3:0x0].0x0 (ffff960e83c07d00) 00010000:00010000:3.0:1785220585.758042:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758045:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c7aac00/0xf62499ab924d1de9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1a3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a74276 expref: 258 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758061:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x982:0x0].0x0 (ffff960d8b9e9200) 00010000:00010000:3.0:1785220585.758064:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758068:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a258600/0xf62499ab9270ecaf lrc: 0/0,0 mode: --/CR res: [0x240000403:0x982:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b508bd expref: 257 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758111:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x289:0x0].0x0 (ffff960d892b9f00) 00010000:00010000:3.0:1785220585.758114:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758118:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87e20800/0xf62499ab924f8dbb lrc: 0/0,0 mode: --/CR res: [0x240000402:0x289:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a81166 expref: 256 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758133:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x33e:0x0].0x0 (ffff960e829ca700) 00010000:00010000:3.0:1785220585.758135:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758139:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e6e00/0xf62499ab925387b6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x33e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9835c expref: 255 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758153:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x31a:0x0].0x0 (ffff960d89e82c00) 00010000:00010000:3.0:1785220585.758156:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758159:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88fd9400/0xf62499ab925187dd lrc: 0/0,0 mode: --/CR res: [0x240000403:0x31a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8dc7b expref: 254 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758171:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa1b:0x0].0x0 (ffff960d959b8900) 00010000:00010000:3.0:1785220585.758173:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758177:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d999f9400/0xf62499ab92741329 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa1b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b65c59 expref: 253 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758191:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaf6:0x0].0x0 (ffff960d9750e000) 00010000:00010000:3.0:1785220585.758193:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758197:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b0eb800/0xf62499ab9278c8aa lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaf6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dc8e expref: 252 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758212:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x307:0x0].0x0 (ffff960eba97e900) 00010000:00010000:3.0:1785220585.758215:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758219:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cd000/0xf62499ab92514938 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x307:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8c616 expref: 251 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758231:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa52:0x0].0x0 (ffff960d8fd32700) 00010000:00010000:3.0:1785220585.758233:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758237:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d91628000/0xf62499ab927556aa lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6c7b3 expref: 250 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758253:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2b2:0x0].0x0 (ffff960e86f6ab00) 00010000:00010000:3.0:1785220585.758255:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758259:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88513a00/0xf62499ab925006dd lrc: 0/0,0 mode: --/CR res: [0x240000402:0x2b2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a85ea5 expref: 249 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758272:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa46:0x0].0x0 (ffff960d9499f200) 00010000:00010000:3.0:1785220585.758274:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758278:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e4cde00/0xf62499ab927517b1 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa46:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6a5bd expref: 248 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758292:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x32b:0x0].0x0 (ffff960d87ba4600) 00010000:00010000:3.0:1785220585.758294:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758298:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2cca00/0xf62499ab9251e592 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x32b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8f946 expref: 247 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758314:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5a3:0x0].0x0 (ffff960d8c69f600) 00010000:00010000:3.0:1785220585.758317:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758321:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8effd400/0xf62499ab92587282 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5a3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab18a6 expref: 246 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758341:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x38f:0x0].0x0 (ffff960d8bf96900) 00010000:00010000:3.0:1785220585.758344:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758348:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d134000/0xf62499ab9252f0d4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x38f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a94e2d expref: 245 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758365:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9c0:0x0].0x0 (ffff960d8f400f00) 00010000:00010000:3.0:1785220585.758367:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.758371:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93cdaa00/0xf62499ab92721510 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9c0:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5a27e expref: 244 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758385:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x479:0x0].0x0 (ffff960d87ba4400) 00010000:00010000:3.0:1785220585.758388:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758392:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a438000/0xf62499ab9255f424 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x479:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa4ca3 expref: 243 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758403:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x6db:0x0].0x0 (ffff960d9162e700) 00010000:00010000:3.0:1785220585.758405:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758409:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93eb3000/0xf62499ab925d2bfa lrc: 0/0,0 mode: --/CR res: [0x240000403:0x6db:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad6015 expref: 242 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758422:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8d6:0x0].0x0 (ffff960d8a928300) 00010000:00010000:3.0:1785220585.758425:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.758429:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90e1c200/0xf62499ab926b66d9 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x8d6:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b30915 expref: 241 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758443:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x99d:0x0].0x0 (ffff960d8b638800) 00010000:00010000:3.0:1785220585.758446:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758449:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f2ba000/0xf62499ab9271fb16 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x99d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5972c expref: 240 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758463:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x48e:0x0].0x0 (ffff960d8c221600) 00010000:00010000:3.0:1785220585.758465:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758469:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e0600/0xf62499ab9255dad9 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x48e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa452c expref: 239 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758485:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5b1:0x0].0x0 (ffff960d8bf96d00) 00010000:00010000:3.0:1785220585.758488:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758492:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a38c000/0xf62499ab92587d8e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5b1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1bd2 expref: 238 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758506:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x30a:0x0].0x0 (ffff960e87ac1d00) 00010000:00010000:3.0:1785220585.758509:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758512:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfb8d800/0xf62499ab92515818 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x30a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8cbbf expref: 237 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758528:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x316:0x0].0x0 (ffff960d89fe5300) 00010000:00010000:3.0:1785220585.758530:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758534:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ec20cda00/0xf62499ab925177d7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x316:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8d750 expref: 236 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758550:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3d7:0x0].0x0 (ffff960e82a72b00) 00010000:00010000:3.0:1785220585.758553:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1785220585.758557:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cb800/0xf62499ab92538cef lrc: 0/0,0 mode: --/PW res: [0x240000402:0x3d7:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a984ba expref: 235 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758573:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x859:0x0].0x0 (ffff960d94f4d000) 00010000:00010000:3.0:1785220585.758576:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758580:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9ae0f200/0xf62499ab92675e7c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x859:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b14d1a expref: 234 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758594:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x739:0x0].0x0 (ffff960d8b2ffe00) 00010000:00010000:3.0:1785220585.758597:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758600:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97910800/0xf62499ab925efb35 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x739:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae5753 expref: 233 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758614:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x39f:0x0].0x0 (ffff960d8c220a00) 00010000:00010000:3.0:1785220585.758616:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.758620:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93403600/0xf62499ab92595e4b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x39f:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab59d6 expref: 232 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758635:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xae7:0x0].0x0 (ffff960d912c9d00) 00010000:00010000:3.0:1785220585.758638:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758641:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a98f200/0xf62499ab9278adc2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xae7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7cfec expref: 231 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758653:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x596:0x0].0x0 (ffff960d8f62c800) 00010000:00010000:3.0:1785220585.758656:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758660:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f821c00/0xf62499ab9258567b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x596:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab0fd1 expref: 230 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758673:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x94a:0x0].0x0 (ffff960d9488a500) 00010000:00010000:3.0:1785220585.758676:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758680:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9643fc00/0xf62499ab926e9ac0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x94a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b4455e expref: 229 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758710:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x662:0x0].0x0 (ffff960d9140e300) 00010000:00010000:3.0:1785220585.758713:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758716:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c5e5c00/0xf62499ab925990fd lrc: 0/0,0 mode: --/CR res: [0x240000403:0x662:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab6b80 expref: 228 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758735:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x325:0x0].0x0 (ffff960d8aee6b00) 00010000:00010000:3.0:1785220585.758737:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758740:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d884aa200/0xf62499ab9251ad7d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x325:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8e80c expref: 227 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758753:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x88a:0x0].0x0 (ffff960d98c22000) 00010000:00010000:3.0:1785220585.758756:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758759:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95a7b000/0xf62499ab9268f58d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x88a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b1c604 expref: 226 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758773:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7e9:0x0].0x0 (ffff960d9795f900) 00010000:00010000:3.0:1785220585.758775:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758779:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b069a00/0xf62499ab92624a9e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x7e9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af7630 expref: 225 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758793:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x883:0x0].0x0 (ffff960d8cb2a300) 00010000:00010000:3.0:1785220585.758795:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.758799:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d897bc800/0xf62499ab92729292 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x883:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d86a expref: 224 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758816:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x58:0x0].0x0 (ffff960eacc3bd00) 00010000:00010000:3.0:1785220585.758818:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758822:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ea68b5200/0xf62499ab92497229 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x58:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a6051c expref: 223 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758835:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5c6:0x0].0x0 (ffff960d810afa00) 00010000:00010000:3.0:1785220585.758837:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758841:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9155ca00/0xf62499ab92589b9b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5c6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab24e6 expref: 222 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758857:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x703:0x0].0x0 (ffff960d94a75300) 00010000:00010000:3.0:1785220585.758859:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758862:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d967f2200/0xf62499ab925db1da lrc: 0/0,0 mode: --/CR res: [0x240000403:0x703:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad9552 expref: 221 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758873:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3d7:0x0].0x0 (ffff960e82a72b00) 00010000:00010000:3.0:1785220585.758875:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758878:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8db6d000/0xf62499ab9257ac05 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x3d7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aad6c7 expref: 220 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758893:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1fa:0x0].0x0 (ffff960ea4cd7100) 00010000:00010000:3.0:1785220585.758895:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758898:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d892b7800/0xf62499ab924e6bf1 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1fa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7ad9f expref: 219 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758911:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x413:0x0].0x0 (ffff960d8ab88800) 00010000:00010000:3.0:1785220585.758914:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758917:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d134200/0xf62499ab925571d9 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x413:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa21e6 expref: 218 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758928:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x698:0x0].0x0 (ffff960d901e1300) 00010000:00010000:3.0:1785220585.758930:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758933:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92228000/0xf62499ab925ab9f8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x698:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ac0254 expref: 217 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758947:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x31d:0x0].0x0 (ffff960e86f6aa00) 00010000:00010000:3.0:1785220585.758950:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1785220585.758953:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d86a4e400/0xf62499ab92515292 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x31d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8c9c0 expref: 216 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758964:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4a9:0x0].0x0 (ffff960d8cc44700) 00010000:00010000:3.0:1785220585.758967:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.758970:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ee06800/0xf62499ab9257e268 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x4a9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aae920 expref: 215 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.758981:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9bd:0x0].0x0 (ffff960d8d06d500) 00010000:00010000:3.0:1785220585.758983:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.758986:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d99990600/0xf62499ab927205a4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9bd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b59bf5 expref: 214 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759001:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9bb:0x0].0x0 (ffff960d93652700) 00010000:00010000:3.0:1785220585.759003:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759006:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d96512000/0xf62499ab9271fcf2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9bb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5981a expref: 213 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759023:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x480:0x0].0x0 (ffff960d87ba3f00) 00010000:00010000:3.0:1785220585.759025:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759027:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d5c400/0xf62499ab92559916 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x480:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa2f76 expref: 212 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759041:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3ff:0x0].0x0 (ffff960d8da0de00) 00010000:00010000:3.0:1785220585.759043:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759046:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d0de00/0xf62499ab925478fe lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3ff:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9d318 expref: 211 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759060:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x906:0x0].0x0 (ffff960d97466400) 00010000:00010000:3.0:1785220585.759062:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759065:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a102600/0xf62499ab926c41fb lrc: 0/0,0 mode: --/CR res: [0x240000403:0x906:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b344aa expref: 210 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759080:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x199:0x0].0x0 (ffff960d8bc2b800) 00010000:00010000:3.0:1785220585.759082:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759108:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3fc00/0xf62499ab924d6d7b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x199:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75b35 expref: 209 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759123:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa9c:0x0].0x0 (ffff960d912c6800) 00010000:00010000:3.0:1785220585.759125:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759128:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9019f600/0xf62499ab9277f6fe lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77818 expref: 208 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759144:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.759146:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.759150:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88510e00/0xf62499ab9249aa30 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a61904 expref: 207 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759161:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x36d:0x0].0x0 (ffff960d8d983d00) 00010000:00010000:3.0:1785220585.759163:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint a 00010000:00010000:3.0:1785220585.759167:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3da00/0xf62499ab925293b2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x36d:0x0].0x0 bits 0xa/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a93051 expref: 206 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759176:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.759178:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1785220585.759181:0:10345:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759190:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x988:0x0].0x0 (ffff960d9986d900) 00010000:00010000:3.0:1785220585.759192:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759195:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94cb3400/0xf62499ab9270fc76 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x988:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b50e74 expref: 205 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759209:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x77b:0x0].0x0 (ffff960d8e8e5500) 00010000:00010000:3.0:1785220585.759211:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759214:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d959ece00/0xf62499ab926046ab lrc: 0/0,0 mode: --/CR res: [0x240000403:0x77b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aedc3e expref: 204 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759227:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.759229:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.759231:0:10345:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759240:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2d5:0x0].0x0 (ffff960d8a4a2400) 00010000:00010000:3.0:1785220585.759242:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759245:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c19fc00/0xf62499ab92519073 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x2d5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8de96 expref: 203 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759258:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8d3:0x0].0x0 (ffff960e812e3200) 00010000:00010000:3.0:1785220585.759261:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759265:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e805600/0xf62499ab926b5846 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x8d3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b30548 expref: 202 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759281:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9a0:0x0].0x0 (ffff960d9ae65a00) 00010000:00010000:3.0:1785220585.759284:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759288:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8eccc200/0xf62499ab92719bd2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9a0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56dfe expref: 201 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759302:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x511:0x0].0x0 (ffff960d8f483800) 00010000:00010000:3.0:1785220585.759304:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759308:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8db69800/0xf62499ab92573c3d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x511:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aab3b2 expref: 200 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759319:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x17:0x0].0x0 (ffff960eba97ee00) 00010000:00010000:3.0:1785220585.759321:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759324:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89158200/0xf62499ab92490452 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x17:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a5dd7d expref: 199 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759336:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x950:0x0].0x0 (ffff960d8f29b000) 00010000:00010000:3.0:1785220585.759338:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759342:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cfe0e00/0xf62499ab926eaaf0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x950:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b44978 expref: 198 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759362:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3d4:0x0].0x0 (ffff960eba97e800) 00010000:00010000:3.0:1785220585.759364:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.759368:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89de5c00/0xf62499ab92542464 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x3d4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9b829 expref: 197 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759384:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x975:0x0].0x0 (ffff960d90440300) 00010000:00010000:3.0:1785220585.759386:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.759390:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90e41c00/0xf62499ab92727a19 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x975:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5cd88 expref: 196 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759404:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7de:0x0].0x0 (ffff960d94e17500) 00010000:00010000:3.0:1785220585.759406:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.759410:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c19600/0xf62499ab9265bcd6 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7de:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b0637a expref: 195 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759434:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x768:0x0].0x0 (ffff960d90015e00) 00010000:00010000:3.0:1785220585.759436:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.759440:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9777aa00/0xf62499ab92723dab lrc: 0/0,0 mode: --/PR res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5b3f7 expref: 194 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759455:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2ee:0x0].0x0 (ffff960d8c1ecf00) 00010000:00010000:3.0:1785220585.759458:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759461:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9fe400/0xf62499ab92510f32 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2ee:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8b4ea expref: 193 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759475:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3fb:0x0].0x0 (ffff960d8c220d00) 00010000:00010000:3.0:1785220585.759477:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759480:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2c5600/0xf62499ab9253eea2 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3fb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9a521 expref: 192 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759492:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x177:0x0].0x0 (ffff960d8a12c300) 00010000:00010000:3.0:1785220585.759494:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759497:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cb200/0xf62499ab924cc6af lrc: 0/0,0 mode: --/CR res: [0x240000403:0x177:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a72588 expref: 191 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759511:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x21:0x0].0x0 (ffff960d8ce3f300) 00010000:00010000:3.0:1785220585.759513:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.759515:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d967e9800/0xf62499ab925e2426 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1adbebf expref: 190 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759524:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x44b:0x0].0x0 (ffff960ec20ff000) 00010000:00010000:3.0:1785220585.759526:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759529:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a276e00/0xf62499ab92584724 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x44b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab0bc5 expref: 189 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759542:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9ae:0x0].0x0 (ffff960d95b3f800) 00010000:00010000:3.0:1785220585.759544:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.759547:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e800/0xf62499ab92727232 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9ae:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ca2b expref: 188 pid: 8450 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759559:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4d3:0x0].0x0 (ffff960e9a78ab00) 00010000:00010000:3.0:1785220585.759562:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759565:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c19e200/0xf62499ab92567f13 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4d3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa782b expref: 187 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759586:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x63c:0x0].0x0 (ffff960d8ddc5d00) 00010000:00010000:3.0:1785220585.759589:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759592:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e6a00/0xf62499ab9258fbf0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x63c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab4340 expref: 186 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759605:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.759607:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1785220585.759610:0:10345:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759620:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x253:0x0].0x0 (ffff960eba97e500) 00010000:00010000:3.0:1785220585.759622:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759625:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87f06e00/0xf62499ab924f4044 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x253:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7f8d1 expref: 185 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759639:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf6:0x0].0x0 (ffff960d8aee6000) 00010000:00010000:3.0:1785220585.759641:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.759644:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a439e00/0xf62499ab924c4058 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xf6:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a6f92e expref: 184 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759658:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x711:0x0].0x0 (ffff960d8daa1800) 00010000:00010000:3.0:1785220585.759660:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759664:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e4e0e00/0xf62499ab925dbce6 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x711:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad9a1b expref: 183 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.759677:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xa70:0x0].0x0 (ffff960ebd94fd00) 00010000:00010000:3.0:1785220585.759679:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.759683:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97affc00/0xf62499ab9278717e lrc: 0/0,0 mode: --/CR res: [0x240000402:0xa70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b1a0 expref: 182 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.761589:0:7470:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8a39a3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1785220585.761661:0:13878:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1785220585.761917:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa9e:0x0].0x0 (ffff960d8f633900) 00010000:00010000:3.0:1785220585.761921:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.761931:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d914d0a00/0xf62499ab9277fbf8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b77a64 expref: 181 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.761951:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x939:0x0].0x0 (ffff960d89bf2100) 00010000:00010000:3.0:1785220585.761953:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.761957:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94ad7000/0xf62499ab926e0ece lrc: 0/0,0 mode: --/PR res: [0x240000403:0x939:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b42425 expref: 180 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.761967:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9da:0x0].0x0 (ffff960d98544f00) 00010000:00010000:3.0:1785220585.761969:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.761973:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89e91e00/0xf62499ab92739c30 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x9da:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b63c0e expref: 179 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.761989:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa4e:0x0].0x0 (ffff960d90cf6600) 00010000:00010000:3.0:1785220585.761991:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.761994:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea94600/0xf62499ab927537d2 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xa4e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6aea0 expref: 178 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762005:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x302:0x0].0x0 (ffff960d8bf96f00) 00010000:00010000:3.0:1785220585.762007:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762010:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d5d400/0xf62499ab925140a2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x302:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8c337 expref: 177 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762026:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x77f:0x0].0x0 (ffff960d93d30000) 00010000:00010000:3.0:1785220585.762029:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762032:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f13d600/0xf62499ab92605f6a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x77f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aee265 expref: 176 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762046:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2f6:0x0].0x0 (ffff960e86f6af00) 00010000:00010000:3.0:1785220585.762048:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762051:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9fe600/0xf62499ab92512597 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2f6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8bbc0 expref: 175 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762065:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3a2:0x0].0x0 (ffff960ea4cd7c00) 00010000:00010000:3.0:1785220585.762067:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762070:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d03a600/0xf62499ab92533bf8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3a2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9692a expref: 174 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762082:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x811:0x0].0x0 (ffff960d8b232e00) 00010000:00010000:3.0:1785220585.762096:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762156:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94d7aa00/0xf62499ab926330cc lrc: 0/0,0 mode: --/CR res: [0x240000402:0x811:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afdd3f expref: 173 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762177:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x10f:0x0].0x0 (ffff960e839f7600) 00010000:00010000:3.0:1785220585.762179:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762183:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c11e00/0xf62499ab924cfb44 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x10f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a73701 expref: 172 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762199:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3cd:0x0].0x0 (ffff960e82a72800) 00010000:00010000:3.0:1785220585.762202:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762205:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2cd600/0xf62499ab92539caf lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3cd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a989c2 expref: 171 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762226:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x581:0x0].0x0 (ffff960d89ff2700) 00010000:00010000:3.0:1785220585.762228:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762231:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a438400/0xf62499ab92582f4c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x581:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab036e expref: 170 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762247:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x61c:0x0].0x0 (ffff960d8cea5200) 00010000:00010000:3.0:1785220585.762250:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762253:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c5e6000/0xf62499ab9258dee6 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x61c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab3aaa expref: 169 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762280:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x665:0x0].0x0 (ffff960d901dc500) 00010000:00010000:3.0:1785220585.762283:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762287:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92380600/0xf62499ab9259a3f7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x665:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab71e6 expref: 168 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762312:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x86d:0x0].0x0 (ffff960d96734400) 00010000:00010000:3.0:1785220585.762314:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762318:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97915600/0xf62499ab92686136 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x86d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b198ca expref: 167 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762333:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x13d:0x0].0x0 (ffff960d8a4a2a00) 00010000:00010000:3.0:1785220585.762335:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762339:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87e20a00/0xf62499ab924d3527 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x13d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7491b expref: 166 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762354:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1be:0x0].0x0 (ffff960d8a12c400) 00010000:00010000:3.0:1785220585.762357:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762360:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc3b400/0xf62499ab924d7b12 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1be:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75f10 expref: 165 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762373:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x386:0x0].0x0 (ffff960d810afc00) 00010000:00010000:3.0:1785220585.762376:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.762379:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c5e6400/0xf62499ab9258c0e0 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x386:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab2f58 expref: 164 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762392:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x212:0x0].0x0 (ffff960e9ac51200) 00010000:00010000:3.0:1785220585.762395:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.762398:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9ff800/0xf62499ab924e9ec6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x212:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7bf88 expref: 163 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762413:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x67f:0x0].0x0 (ffff960e86a82200) 00010000:00010000:3.0:1785220585.762416:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762420:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960e86a55800/0xf62499ab9259d456 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x67f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab8064 expref: 162 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762433:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x47f:0x0].0x0 (ffff960d9061ca00) 00010000:00010000:3.0:1785220585.762436:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762439:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8baf3a00/0xf62499ab9255905d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x47f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa2c19 expref: 161 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762452:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa73:0x0].0x0 (ffff960d94ff8f00) 00010000:00010000:3.0:1785220585.762454:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762457:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97787400/0xf62499ab92763f39 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b708f1 expref: 160 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762476:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4bf:0x0].0x0 (ffff960d89376d00) 00010000:00010000:3.0:1785220585.762478:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762482:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e6000/0xf62499ab925648a2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4bf:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa6681 expref: 159 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762497:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x808:0x0].0x0 (ffff960d954dbd00) 00010000:00010000:3.0:1785220585.762499:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762503:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d76ba00/0xf62499ab92635523 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x808:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afe6c3 expref: 158 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762516:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2b2:0x0].0x0 (ffff960d8c1ecc00) 00010000:00010000:3.0:1785220585.762518:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762522:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebdb2be00/0xf62499ab9250d698 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x2b2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8a30f expref: 157 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762536:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9cb:0x0].0x0 (ffff960d8f3ad900) 00010000:00010000:3.0:1785220585.762538:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.762541:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d954c4400/0xf62499ab92723e37 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9cb:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5b436 expref: 156 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762555:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x47b:0x0].0x0 (ffff960eacc3eb00) 00010000:00010000:3.0:1785220585.762557:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762560:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d86a4f200/0xf62499ab92553c9c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x47b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa1169 expref: 155 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762574:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4a2:0x0].0x0 (ffff960d8ea99b00) 00010000:00010000:3.0:1785220585.762577:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762581:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f821a00/0xf62499ab9255f328 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4a2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa4c80 expref: 154 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762594:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x738:0x0].0x0 (ffff960d8f432e00) 00010000:00010000:3.0:1785220585.762596:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762599:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ecf5200/0xf62499ab925f1e19 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x738:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae661e expref: 153 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762614:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x984:0x0].0x0 (ffff960d92192b00) 00010000:00010000:3.0:1785220585.762617:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762620:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a32a400/0xf62499ab92720366 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x984:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b59ae4 expref: 152 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762633:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x19b:0x0].0x0 (ffff960d87ba5600) 00010000:00010000:3.0:1785220585.762636:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762639:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f49400/0xf62499ab924d0466 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x19b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7394d expref: 151 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762651:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x479:0x0].0x0 (ffff960d87ba4400) 00010000:00010000:3.0:1785220585.762653:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762656:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ce60600/0xf62499ab9255d028 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x479:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa412e expref: 150 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762674:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x819:0x0].0x0 (ffff960d8be7dd00) 00010000:00010000:3.0:1785220585.762676:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.762680:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9b117c00/0xf62499ab92676893 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x819:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b15093 expref: 149 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762710:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa01:0x0].0x0 (ffff960d95643e00) 00010000:00010000:3.0:1785220585.762712:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762716:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d91568400/0xf62499ab9273658e lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa01:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b62ebd expref: 148 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762729:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4ce:0x0].0x0 (ffff960e83c07c00) 00010000:00010000:3.0:1785220585.762731:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.762734:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9155e600/0xf62499ab92592be6 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x4ce:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab4f10 expref: 147 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762746:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9fa:0x0].0x0 (ffff960d906c9200) 00010000:00010000:3.0:1785220585.762748:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.762751:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90196e00/0xf62499ab9274834c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x9fa:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b683a4 expref: 146 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762879:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3c0:0x0].0x0 (ffff960eba991800) 00010000:00010000:3.0:1785220585.762882:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.762902:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d890bcc00/0xf62499ab92537f04 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3c0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a980bc expref: 145 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.762993:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x430:0x0].0x0 (ffff960d86fd2200) 00010000:00010000:3.0:1785220585.762996:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763000:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d87e20e00/0xf62499ab92549f85 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x430:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9e134 expref: 144 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763018:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa48:0x0].0x0 (ffff960d8b206600) 00010000:00010000:3.0:1785220585.763021:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763024:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9156be00/0xf62499ab92752166 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa48:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6a7d8 expref: 143 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763037:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7e4:0x0].0x0 (ffff960d9065a700) 00010000:00010000:3.0:1785220585.763040:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763044:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c5c2800/0xf62499ab92624128 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7e4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af7351 expref: 142 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763058:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xa80:0x0].0x0 (ffff960d8f8a7c00) 00010000:00010000:3.0:1785220585.763060:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.763063:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93686000/0xf62499ab9278b4b4 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xa80:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d365 expref: 141 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763075:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x43a:0x0].0x0 (ffff960d87ba5400) 00010000:00010000:3.0:1785220585.763078:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763082:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c01ac00/0xf62499ab9254b40e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x43a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9e7fc expref: 140 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763195:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff960eba8e7100) 00010000:00010000:3.0:1785220585.763198:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763202:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e6800/0xf62499ab924abe5c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a67451 expref: 139 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763217:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2eb:0x0].0x0 (ffff960eb7389b00) 00010000:00010000:3.0:1785220585.763219:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763222:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89072c00/0xf62499ab92520e49 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x2eb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a90643 expref: 138 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763242:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x97d:0x0].0x0 (ffff960d9012a900) 00010000:00010000:3.0:1785220585.763244:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.763247:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b069600/0xf62499ab92720901 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x97d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b59d61 expref: 137 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763262:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa35:0x0].0x0 (ffff960d91298700) 00010000:00010000:3.0:1785220585.763265:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763268:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d914da200/0xf62499ab927489ce lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b68621 expref: 136 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763284:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa2e:0x0].0x0 (ffff960d89ff5300) 00010000:00010000:3.0:1785220585.763286:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763290:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ab64800/0xf62499ab9274708a lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa2e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b67a90 expref: 135 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763304:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9a2:0x0].0x0 (ffff960d8f368c00) 00010000:00010000:3.0:1785220585.763307:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763310:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ff89800/0xf62499ab9271a270 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9a2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b570c8 expref: 134 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763325:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x28:0x0].0x0 (ffff960d96672300) 00010000:00010000:3.0:1785220585.763327:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.763330:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90720800/0xf62499ab9261bfd9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x28:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af4dbf expref: 133 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763341:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x980:0x0].0x0 (ffff960d8dcca300) 00010000:00010000:3.0:1785220585.763344:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763347:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9929b200/0xf62499ab9270e28a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x980:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b504fe expref: 132 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763362:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x34b:0x0].0x0 (ffff960d87ba6000) 00010000:00010000:3.0:1785220585.763365:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763369:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d66e00/0xf62499ab92540c4d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x34b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9af62 expref: 131 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763388:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x867:0x0].0x0 (ffff960d8e8aa200) 00010000:00010000:3.0:1785220585.763390:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763394:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9351fc00/0xf62499ab92682a40 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x867:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b18db7 expref: 130 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763408:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4ce:0x0].0x0 (ffff960d8b294300) 00010000:00010000:3.0:1785220585.763410:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763413:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8fe05600/0xf62499ab92567a4a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4ce:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa76db expref: 129 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763426:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x703:0x0].0x0 (ffff960d94a75300) 00010000:00010000:3.0:1785220585.763429:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.763432:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9572a000/0xf62499ab925db075 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x703:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad94b1 expref: 128 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763446:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x77e:0x0].0x0 (ffff960d8b340d00) 00010000:00010000:3.0:1785220585.763448:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763451:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93c9bc00/0xf62499ab92617bb5 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x77e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af3a08 expref: 127 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763471:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa43:0x0].0x0 (ffff960d909d5900) 00010000:00010000:3.0:1785220585.763473:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763476:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d96676400/0xf62499ab92750574 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa43:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6a22f expref: 126 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763494:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1ba:0x0].0x0 (ffff960d86fd2500) 00010000:00010000:3.0:1785220585.763496:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763500:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6cca00/0xf62499ab924d708b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1ba:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75bf2 expref: 125 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763513:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x575:0x0].0x0 (ffff960d8ea99f00) 00010000:00010000:3.0:1785220585.763515:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763519:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9c0000/0xf62499ab92580bff lrc: 0/0,0 mode: --/CR res: [0x240000403:0x575:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aaf7ac expref: 124 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763534:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x404:0x0].0x0 (ffff960d8c221500) 00010000:00010000:3.0:1785220585.763536:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763540:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3d800/0xf62499ab9253fd9e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x404:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9ab17 expref: 123 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763552:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3ca:0x0].0x0 (ffff960d810afe00) 00010000:00010000:3.0:1785220585.763554:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763558:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a439600/0xf62499ab92536d45 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3ca:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a97a64 expref: 122 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763569:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9e7:0x0].0x0 (ffff960d94e09800) 00010000:00010000:3.0:1785220585.763572:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1785220585.763576:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94e46600/0xf62499ab92728f5f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d6aa expref: 121 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763598:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3f0:0x0].0x0 (ffff960d8a12ca00) 00010000:00010000:3.0:1785220585.763601:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763604:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f9ed800/0xf62499ab9253e2ee lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3f0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9a2b2 expref: 120 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763618:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7d7:0x0].0x0 (ffff960e82ba6300) 00010000:00010000:3.0:1785220585.763621:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763624:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfb58c00/0xf62499ab9261e0b0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7d7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1af5743 expref: 119 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763639:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1fa:0x0].0x0 (ffff960ea4cd7100) 00010000:00010000:3.0:1785220585.763641:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763644:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89a5ca00/0xf62499ab924e3b68 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1fa:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a79d22 expref: 118 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763660:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x326:0x0].0x0 (ffff960e8ab7ef00) 00010000:00010000:3.0:1785220585.763663:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763666:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a275000/0xf62499ab9252e46a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x326:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a94a2f expref: 117 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763680:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x821:0x0].0x0 (ffff960d89842a00) 00010000:00010000:3.0:1785220585.763682:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763685:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c717800/0xf62499ab92642842 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x821:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b016c7 expref: 116 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763718:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x413:0x0].0x0 (ffff960d8ab88800) 00010000:00010000:3.0:1785220585.763721:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763724:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e64c200/0xf62499ab9254562f lrc: 0/0,0 mode: --/CR res: [0x240000403:0x413:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9c852 expref: 115 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763740:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x680:0x0].0x0 (ffff960d89bf5a00) 00010000:00010000:3.0:1785220585.763743:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763747:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92380400/0xf62499ab9259d31b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x680:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab7fc3 expref: 114 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763836:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x69a:0x0].0x0 (ffff960d87d34a00) 00010000:00010000:3.0:1785220585.763839:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763844:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90528400/0xf62499ab925ab5e5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x69a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ac0182 expref: 113 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763862:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6bf:0x0].0x0 (ffff960d8eaf8600) 00010000:00010000:3.0:1785220585.763864:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.763868:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40b600/0xf62499ab925e5eb8 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6bf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ade5cb expref: 112 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763884:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x30c:0x0].0x0 (ffff960e88c64700) 00010000:00010000:3.0:1785220585.763886:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763889:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9ff600/0xf62499ab925159b5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x30c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8ccc9 expref: 111 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763900:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa7e:0x0].0x0 (ffff960d8d6f4900) 00010000:00010000:3.0:1785220585.763903:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.763906:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93616e00/0xf62499ab9276cfe6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xa7e:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74455 expref: 110 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763917:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa4e:0x0].0x0 (ffff960d90cf6600) 00010000:00010000:3.0:1785220585.763920:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.763923:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b1cac00/0xf62499ab92761ff8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa4e:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b70563 expref: 109 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763937:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xab7:0x0].0x0 (ffff960d9215fb00) 00010000:00010000:3.0:1785220585.763939:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763942:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8da26e00/0xf62499ab92782c3b lrc: 0/0,0 mode: --/CR res: [0x240000403:0xab7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79186 expref: 108 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763954:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa4d:0x0].0x0 (ffff960d97736d00) 00010000:00010000:3.0:1785220585.763957:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.763960:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d958faa00/0xf62499ab92753237 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xa4d:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6ace0 expref: 107 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763972:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1b1:0x0].0x0 (ffff960d8a4a2d00) 00010000:00010000:3.0:1785220585.763975:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763978:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2c5800/0xf62499ab924d4a6d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1b1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a74f81 expref: 106 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.763991:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x52f:0x0].0x0 (ffff960e86337700) 00010000:00010000:3.0:1785220585.763993:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.763997:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9096be00/0xf62499ab925862ec lrc: 0/0,0 mode: --/CR res: [0x240000402:0x52f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab1438 expref: 105 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764012:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x391:0x0].0x0 (ffff960d8bf96e00) 00010000:00010000:3.0:1785220585.764014:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764017:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b1e1400/0xf62499ab9252ec82 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x391:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a94d9a expref: 104 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764035:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3ee:0x0].0x0 (ffff960ead0e5600) 00010000:00010000:3.0:1785220585.764037:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764041:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d906e6200/0xf62499ab92546b21 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3ee:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9cf59 expref: 103 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764054:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x454:0x0].0x0 (ffff960d86fd2d00) 00010000:00010000:3.0:1785220585.764056:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764059:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f3cbe00/0xf62499ab9254f46c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x454:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9fb43 expref: 102 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764073:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa83:0x0].0x0 (ffff960d90104d00) 00010000:00010000:3.0:1785220585.764075:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764078:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97545000/0xf62499ab9276ee7f lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b74b5c expref: 101 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764302:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9be:0x0].0x0 (ffff960d8c22a000) 00010000:00010000:3.0:1785220585.764305:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764310:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9573b400/0xf62499ab92720b3f lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9be:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b59e4f expref: 100 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764326:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5ce:0x0].0x0 (ffff960d909fa300) 00010000:00010000:3.0:1785220585.764329:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.764333:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d915be600/0xf62499ab9259c300 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x5ce:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab7b78 expref: 99 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764348:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa99:0x0].0x0 (ffff960d9902dc00) 00010000:00010000:3.0:1785220585.764350:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764354:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9096b000/0xf62499ab9277dd20 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7736b expref: 98 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764367:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x515:0x0].0x0 (ffff960d906e2000) 00010000:00010000:3.0:1785220585.764369:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764372:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9096b400/0xf62499ab92586e99 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x515:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab17a3 expref: 97 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764386:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9a7:0x0].0x0 (ffff960d81d9e800) 00010000:00010000:3.0:1785220585.764389:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.764393:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8dcc1a00/0xf62499ab92725f4d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x9a7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5c2c2 expref: 96 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764404:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x939:0x0].0x0 (ffff960d89bf2100) 00010000:00010000:3.0:1785220585.764407:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764410:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e214a00/0xf62499ab926e11e5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x939:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b42521 expref: 95 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764424:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x49f:0x0].0x0 (ffff960d86cfb300) 00010000:00010000:3.0:1785220585.764426:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764430:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88fd8a00/0xf62499ab9255e392 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x49f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa47c5 expref: 94 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764449:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa9d:0x0].0x0 (ffff960d8dd54e00) 00010000:00010000:3.0:1785220585.764451:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.764454:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94ac0000/0xf62499ab927841ea lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa9d:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b79bb9 expref: 93 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764465:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3a9:0x0].0x0 (ffff960d810af900) 00010000:00010000:3.0:1785220585.764467:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764471:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ddae600/0xf62499ab925349ce lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3a9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a96dc2 expref: 92 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764484:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x7fb:0x0].0x0 (ffff960ebda9f900) 00010000:00010000:3.0:1785220585.764486:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764489:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89d66c00/0xf62499ab9262f5d8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x7fb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afcecf expref: 91 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764504:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x272:0x0].0x0 (ffff960d8c218000) 00010000:00010000:3.0:1785220585.764506:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764510:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a341a00/0xf62499ab924f6dc4 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x272:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a806ae expref: 90 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764525:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x44:0x0].0x0 (ffff960d923f8800) 00010000:00010000:3.0:1785220585.764527:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.764531:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9746f400/0xf62499ab926e5a0e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b435dd expref: 89 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764546:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3a5:0x0].0x0 (ffff960eba991700) 00010000:00010000:3.0:1785220585.764548:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764552:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc39000/0xf62499ab92533c7d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3a5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a96977 expref: 88 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764564:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x2d8:0x0].0x0 (ffff960d8c1ed100) 00010000:00010000:3.0:1785220585.764567:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.764570:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc32800/0xf62499ab9250d174 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x2d8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8a14f expref: 87 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764588:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x63f:0x0].0x0 (ffff960d8cfff300) 00010000:00010000:3.0:1785220585.764591:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.764594:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8acc8c00/0xf62499ab92593770 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x63f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab50ad expref: 86 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764610:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x709:0x0].0x0 (ffff960d8b6afc00) 00010000:00010000:3.0:1785220585.764612:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764616:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960e86a54c00/0xf62499ab925da125 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x709:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ad8db8 expref: 85 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764631:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x898:0x0].0x0 (ffff960d9237dd00) 00010000:00010000:3.0:1785220585.764633:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764636:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f42ce00/0xf62499ab926978b8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x898:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b2215f expref: 84 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764659:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x48a:0x0].0x0 (ffff960d8c220700) 00010000:00010000:3.0:1785220585.764661:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764665:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88fdd600/0xf62499ab9255a7da lrc: 0/0,0 mode: --/CR res: [0x240000403:0x48a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa33dd expref: 83 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764685:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x511:0x0].0x0 (ffff960d8f483800) 00010000:00010000:3.0:1785220585.764688:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1785220585.764701:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8aebd400/0xf62499ab92573b25 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x511:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aab1c1 expref: 82 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764716:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x30:0x0].0x0 (ffff960d903a8a00) 00010000:00010000:3.0:1785220585.764719:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.764722:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e0f4800/0xf62499ab92657a72 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x30:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b056e6 expref: 81 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764735:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9a4:0x0].0x0 (ffff960e86021f00) 00010000:00010000:3.0:1785220585.764737:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764741:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c2fcc00/0xf62499ab92727e3a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x9a4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5cf33 expref: 80 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764755:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9a7:0x0].0x0 (ffff960d81d9e800) 00010000:00010000:3.0:1785220585.764757:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764876:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90e40400/0xf62499ab9272059d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x9a7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b59bd2 expref: 79 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764901:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x307:0x0].0x0 (ffff960eba97e900) 00010000:00010000:3.0:1785220585.764903:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.764907:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d137000/0xf62499ab9252092c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x307:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a90532 expref: 78 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764919:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x37:0x0].0x0 (ffff960d967d5a00) 00010000:00010000:3.0:1785220585.764921:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.764925:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b0ebc00/0xf62499ab9267b9ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x37:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b17346 expref: 77 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764936:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xacc:0x0].0x0 (ffff960d979aac00) 00010000:00010000:3.0:1785220585.764939:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764942:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9156b400/0xf62499ab92786d72 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xacc:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b01f expref: 76 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764956:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x35e:0x0].0x0 (ffff960d8bba2100) 00010000:00010000:3.0:1785220585.764959:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.764962:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c72800/0xf62499ab92524298 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x35e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9174c expref: 75 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764976:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xac1:0x0].0x0 (ffff960d97a0ec00) 00010000:00010000:3.0:1785220585.764978:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.764982:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98593a00/0xf62499ab9278d00c lrc: 0/0,0 mode: --/PR res: [0x240000402:0xac1:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7dfe4 expref: 74 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.764993:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9c1:0x0].0x0 (ffff960d8c22ab00) 00010000:00010000:3.0:1785220585.764995:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1785220585.764998:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bb4d800/0xf62499ab92721605 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5a318 expref: 73 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765011:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x696:0x0].0x0 (ffff960d8ea11400) 00010000:00010000:3.0:1785220585.765014:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765017:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d906f7000/0xf62499ab925aa4f1 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x696:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1abfe25 expref: 72 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765039:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaf2:0x0].0x0 (ffff960d8c0e2a00) 00010000:00010000:3.0:1785220585.765042:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765045:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9232e800/0xf62499ab9278bf11 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaf2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d86d expref: 71 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765057:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1be:0x0].0x0 (ffff960d8ce3ee00) 00010000:00010000:3.0:1785220585.765059:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765062:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc30c00/0xf62499ab9251e4c7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1be:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8f8f2 expref: 70 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765077:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa2f:0x0].0x0 (ffff960d8d192c00) 00010000:00010000:3.0:1785220585.765080:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.765083:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93ff2c00/0xf62499ab92747641 lrc: 0/0,0 mode: --/PR res: [0x240000403:0xa2f:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b67cab expref: 69 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765191:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4d8:0x0].0x0 (ffff960d87ba6800) 00010000:00010000:3.0:1785220585.765194:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765199:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebdb2b600/0xf62499ab92568334 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4d8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa7966 expref: 68 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765216:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x143:0x0].0x0 (ffff960eb7389a00) 00010000:00010000:3.0:1785220585.765218:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765222:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e1600/0xf62499ab924c5ca5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x143:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7021f expref: 67 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765238:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x31f:0x0].0x0 (ffff960d89376800) 00010000:00010000:3.0:1785220585.765241:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765245:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c19ec00/0xf62499ab92519145 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x31f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8deb2 expref: 66 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765466:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa95:0x0].0x0 (ffff960d901bc000) 00010000:00010000:3.0:1785220585.765469:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765474:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92235800/0xf62499ab9277a519 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b76c33 expref: 65 pid: 13871 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765489:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9ec:0x0].0x0 (ffff960d9853e400) 00010000:00010000:3.0:1785220585.765492:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765495:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d915bf200/0xf62499ab9274268c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x9ec:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6626b expref: 64 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765507:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5e0:0x0].0x0 (ffff960d8dd54c00) 00010000:00010000:3.0:1785220585.765509:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765513:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9155f200/0xf62499ab9258adca lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5e0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab2a3b expref: 63 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765526:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x796:0x0].0x0 (ffff960d8f57bb00) 00010000:00010000:3.0:1785220585.765529:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765532:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d958f5400/0xf62499ab9260b2e5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x796:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aefd31 expref: 62 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765548:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4ce:0x0].0x0 (ffff960e83c07c00) 00010000:00010000:3.0:1785220585.765550:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765553:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89ac0000/0xf62499ab92592915 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4ce:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab4e6f expref: 61 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765572:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x63a:0x0].0x0 (ffff960e83c07600) 00010000:00010000:3.0:1785220585.765574:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765577:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b9ffc00/0xf62499ab9258fb80 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x63a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab431d expref: 60 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765590:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x26e:0x0].0x0 (ffff960d8a4a3a00) 00010000:00010000:3.0:1785220585.765593:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.765596:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c591000/0xf62499ab9251fd8d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x26e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a90276 expref: 59 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765610:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa6f:0x0].0x0 (ffff960d93652a00) 00010000:00010000:3.0:1785220585.765613:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765616:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9a416e00/0xf62499ab927632cf lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa6f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b707af expref: 58 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765631:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x321:0x0].0x0 (ffff960d8bba2200) 00010000:00010000:3.0:1785220585.765633:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765636:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88c7f000/0xf62499ab92519aad lrc: 0/0,0 mode: --/CR res: [0x240000403:0x321:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8e12f expref: 57 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765650:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x50:0x0].0x0 (ffff960d92376a00) 00010000:00010000:3.0:1785220585.765652:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.765656:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9a6e4c00/0xf62499ab9278e296 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x50:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7e8b9 expref: 56 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765683:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4a9:0x0].0x0 (ffff960d8cc44700) 00010000:00010000:3.0:1785220585.765686:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1785220585.765701:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f563a00/0xf62499ab9257faef lrc: 0/0,0 mode: --/PR res: [0x240000403:0x4a9:0x0].0x0 bits 0x1a/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aaf162 expref: 55 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765716:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x60b:0x0].0x0 (ffff960d8facaa00) 00010000:00010000:3.0:1785220585.765718:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765722:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b5bd600/0xf62499ab9258d52a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x60b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab380a expref: 54 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765735:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x689:0x0].0x0 (ffff960d8f290600) 00010000:00010000:3.0:1785220585.765738:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765741:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88511800/0xf62499ab925a0fb3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x689:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab8cf1 expref: 53 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765756:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x48f:0x0].0x0 (ffff960d8cc44200) 00010000:00010000:3.0:1785220585.765759:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765829:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f3d400/0xf62499ab9255c2b4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x48f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa3c81 expref: 52 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765850:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xa5a:0x0].0x0 (ffff960d94a75400) 00010000:00010000:3.0:1785220585.765853:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.765856:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d937f7000/0xf62499ab9278f1df lrc: 0/0,0 mode: --/CR res: [0x240000402:0xa5a:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7efc0 expref: 51 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765868:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x764:0x0].0x0 (ffff960d8ac4a700) 00010000:00010000:3.0:1785220585.765871:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765874:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97889200/0xf62499ab925fb48b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x764:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aeb56a expref: 50 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765899:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8b2:0x0].0x0 (ffff960d909f8600) 00010000:00010000:3.0:1785220585.765902:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.765905:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93669a00/0xf62499ab926aa9cb lrc: 0/0,0 mode: --/PR res: [0x240000403:0x8b2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b2cf08 expref: 49 pid: 13920 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765920:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3cc:0x0].0x0 (ffff960d8bc0e300) 00010000:00010000:3.0:1785220585.765922:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765926:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c591800/0xf62499ab92545835 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3cc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9c97f expref: 48 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765938:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x51e:0x0].0x0 (ffff960d89376f00) 00010000:00010000:3.0:1785220585.765941:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765945:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9096ba00/0xf62499ab92582471 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x51e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab005e expref: 47 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765957:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xa61:0x0].0x0 (ffff960d8a12fe00) 00010000:00010000:3.0:1785220585.765959:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765963:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bb50a00/0xf62499ab9278304e lrc: 0/0,0 mode: --/CR res: [0x240000402:0xa61:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b793a1 expref: 46 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765976:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x740:0x0].0x0 (ffff960d9077f400) 00010000:00010000:3.0:1785220585.765978:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.765982:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90738400/0xf62499ab925f0a31 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x740:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ae6101 expref: 45 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.765995:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2fa:0x0].0x0 (ffff960d8c94b500) 00010000:00010000:3.0:1785220585.765997:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766000:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c01b800/0xf62499ab92515428 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x2fa:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a8ca1b expref: 44 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766012:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x5d3:0x0].0x0 (ffff960d8ab88300) 00010000:00010000:3.0:1785220585.766014:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766018:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89a5cc00/0xf62499ab9258a6ed lrc: 0/0,0 mode: --/CR res: [0x240000403:0x5d3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab286d expref: 43 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766032:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x36d:0x0].0x0 (ffff960e829cae00) 00010000:00010000:3.0:1785220585.766034:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766038:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d884aa600/0xf62499ab9253fed2 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x36d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9ab64 expref: 42 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766050:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xac8:0x0].0x0 (ffff960d89134600) 00010000:00010000:3.0:1785220585.766052:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766055:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92308400/0xf62499ab92785ea7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xac8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7a8e7 expref: 41 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766067:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x15:0x0].0x0 (ffff960d8c6bed00) 00010000:00010000:3.0:1785220585.766069:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.766073:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d923e8800/0xf62499ab925ba623 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ac99b4 expref: 40 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766174:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x686:0x0].0x0 (ffff960d8e8e5b00) 00010000:00010000:3.0:1785220585.766177:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766180:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9489fc00/0xf62499ab9259f653 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x686:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab8836 expref: 39 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766200:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xaf3:0x0].0x0 (ffff960d8bb57500) 00010000:00010000:3.0:1785220585.766202:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766206:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7ce00/0xf62499ab9278bf49 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xaf3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7d8ac expref: 38 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766218:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9e0:0x0].0x0 (ffff960d8eb29b00) 00010000:00010000:3.0:1785220585.766220:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.766223:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f49b400/0xf62499ab9272876a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9e0:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d31c expref: 37 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766237:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9dc:0x0].0x0 (ffff960d98bd4a00) 00010000:00010000:3.0:1785220585.766239:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766242:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d978e3200/0xf62499ab92727588 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x9dc:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5cba5 expref: 36 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766309:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x865:0x0].0x0 (ffff960d9498ff00) 00010000:00010000:3.0:1785220585.766312:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766316:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93755400/0xf62499ab9267e176 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x865:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b17df7 expref: 35 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766334:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x880:0x0].0x0 (ffff960d93c1c700) 00010000:00010000:3.0:1785220585.766336:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766340:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a026000/0xf62499ab9268d16e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x880:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b1b90e expref: 34 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766356:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xad3:0x0].0x0 (ffff960d9646c800) 00010000:00010000:3.0:1785220585.766359:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766363:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95694200/0xf62499ab92787cad lrc: 0/0,0 mode: --/CR res: [0x240000403:0xad3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7b6f5 expref: 33 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766377:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9af:0x0].0x0 (ffff960d935cb800) 00010000:00010000:3.0:1785220585.766380:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1785220585.766384:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d902ff000/0xf62499ab9271e0d6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9af:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b58c04 expref: 32 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766398:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x24c:0x0].0x0 (ffff960d8bc2b900) 00010000:00010000:3.0:1785220585.766400:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766404:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8b6e5400/0xf62499ab925228a5 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x24c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a90eb6 expref: 31 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766421:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3db:0x0].0x0 (ffff960ea686f500) 00010000:00010000:3.0:1785220585.766423:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.766426:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8da8ca00/0xf62499ab92550c7c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x3db:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa02c8 expref: 30 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766440:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x825:0x0].0x0 (ffff960d957bd200) 00010000:00010000:3.0:1785220585.766442:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766446:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94ca2c00/0xf62499ab92645fc4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x825:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b021f6 expref: 29 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766461:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x222:0x0].0x0 (ffff960d8bba3f00) 00010000:00010000:3.0:1785220585.766464:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1785220585.766468:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e285400/0xf62499ab92504520 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x222:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a87366 expref: 28 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766488:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0xa62:0x0].0x0 (ffff960d8fde8e00) 00010000:00010000:3.0:1785220585.766490:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766494:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfafa200/0xf62499ab9275bd65 lrc: 0/0,0 mode: --/CR res: [0x240000403:0xa62:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b6f2a8 expref: 27 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766509:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x8de:0x0].0x0 (ffff960d8fdb2100) 00010000:00010000:3.0:1785220585.766511:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1785220585.766515:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d090400/0xf62499ab926dfe6d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x8de:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b41e28 expref: 26 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766529:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x83c:0x0].0x0 (ffff960d95435200) 00010000:00010000:3.0:1785220585.766531:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766534:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95889400/0xf62499ab9265850e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x83c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b05971 expref: 25 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766550:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xa5a:0x0].0x0 (ffff960d94a75400) 00010000:00010000:3.0:1785220585.766553:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.766556:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfc92400/0xf62499ab9278ecc9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xa5a:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7ed82 expref: 24 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766571:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x3b5:0x0].0x0 (ffff960eacc3e000) 00010000:00010000:3.0:1785220585.766573:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766577:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89a5c200/0xf62499ab92535876 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x3b5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a9725a expref: 23 pid: 13806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766590:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x17e:0x0].0x0 (ffff960eba8e7500) 00010000:00010000:3.0:1785220585.766593:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766597:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc33800/0xf62499ab924cd34a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x17e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a729fd expref: 22 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766613:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x2a:0x0].0x0 (ffff960d89153e00) 00010000:00010000:3.0:1785220585.766616:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.766619:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ab64400/0xf62499ab92634594 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1afe294 expref: 21 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766634:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x868:0x0].0x0 (ffff960d87b98900) 00010000:00010000:3.0:1785220585.766636:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766640:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9072e600/0xf62499ab926833af lrc: 0/0,0 mode: --/CR res: [0x240000403:0x868:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b18eb3 expref: 20 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766656:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1a9:0x0].0x0 (ffff960d87b81f00) 00010000:00010000:3.0:1785220585.766659:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766662:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3ca00/0xf62499ab924d37ea lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1a9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a74a95 expref: 19 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766680:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x5ea:0x0].0x0 (ffff960d901d9a00) 00010000:00010000:3.0:1785220585.766683:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766687:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebd940000/0xf62499ab925a19e6 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x5ea:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1ab910b expref: 18 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766714:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9ae:0x0].0x0 (ffff960d95b3f800) 00010000:00010000:3.0:1785220585.766716:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.766720:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8e6a2800/0xf62499ab927270d4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9ae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5c9d7 expref: 17 pid: 8451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766736:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x917:0x0].0x0 (ffff960d91330100) 00010000:00010000:3.0:1785220585.766738:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766741:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d93d76200/0xf62499ab926c9442 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x917:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b3587d expref: 16 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766757:0:10345:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x4c0:0x0].0x0 (ffff960d86cfb200) 00010000:00010000:3.0:1785220585.766879:0:10345:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1785220585.766885:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f563000/0xf62499ab92565019 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x4c0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1aa68aa expref: 15 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.766910:0:10345:0:(ldlm_lock.c:2680:ldlm_export_cancel_locks()) Export ffff960ebd9da800, canceled 535 locks, left on hash table 0. 00010000:00010000:3.0:1785220585.766953:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220585.766955:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220585.766958:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220585.769818:0:13914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510812160 00000100:00100000:1.0:1785220585.769835:0:13914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510812160 00000100:00100000:1.0:1785220585.769904:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12557 00000100:00100000:1.0:1785220585.769914:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7470:x1871938510812160:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1785220585.770075:0:13878:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.770205:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7470:x1871938510812160:12345-0@lo:1000:osp_up0-1.0 Request processed in 291us (22972us total) trans 4295000085 rc 0/0 00000100:00100000:0.0:1785220585.770210:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510812288 00000100:00100000:1.0:1785220585.770212:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12557 00000100:00100000:0.0:1785220585.770215:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510812288 00000100:00100000:0.0:1785220585.770239:0:13815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12558 00000100:00100000:0.0:1785220585.770245:0:13815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13878:x1871938510812288:12345-0@lo:1000:mdt_io00_008.0 00000100:00100000:1.0:1785220585.770284:0:7470:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d921fa680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510812160:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1785220585.770385:0:13815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13878:x1871938510812288:12345-0@lo:1000:mdt_io00_008.0 Request processed in 140us (8725us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.770392:0:13815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12558 00000004:00080000:1.0:1785220585.770414:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000402:0x9fa:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000100:00100000:2.0:1785220585.770459:0:13878:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a5ed880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_008:lustre-MDT0001-mdtlov_UUID:13878:1871938510812288:0@lo:1000:mdt_io00_008.0 00000004:00080000:1.0:1785220585.770548:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000403:0xa4e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00010000:00010000:2.0:1785220585.770556:0:13878:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770567:0:13878:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770575:0:13878:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770581:0:13878:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d949c2800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.770585:0:13878:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770593:0:13878:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770608:0:13878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9b0:0x0].0x0 (ffff960d934bd300) 00010000:00010000:2.0:1785220585.770611:0:13878:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1785220585.770615:0:13878:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 2/0,0 mode: --/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770622:0:13878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d949c2800/0xf62499ab9272733c lrc: 0/0,0 mode: --/EX res: [0x240000402:0x9b0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1785220585.770626:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].1242, 440 off892048 00010000:00010000:2.0:1785220585.770636:0:13878:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673ee00/0xf62499ab92727312 lrc: 3/0,1 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770641:0:13878:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673ee00/0xf62499ab92727312 lrc: 2/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770647:0:13878:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673ee00/0xf62499ab92727312 lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770651:0:13878:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d9673ee00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.770654:0:13878:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673ee00/0xf62499ab92727312 lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770659:0:13878:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673ee00/0xf62499ab92727312 lrc: 4/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770665:0:13878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x806:0x0].0x0 (ffff960d9902d200) 00010000:00010000:2.0:1785220585.770667:0:13878:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1785220585.770670:0:13878:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673ee00/0xf62499ab92727312 lrc: 2/0,0 mode: --/EX res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770675:0:13878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673ee00/0xf62499ab92727312 lrc: 0/0,0 mode: --/EX res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220585.770706:0:13878:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000404:0x806:0x0] 00010000:00010000:2.0:1785220585.770712:0:13878:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 4/0,1 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770720:0:13878:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 4/0,1 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770727:0:13878:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770744:0:13878:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1785220585.770753:0:13878:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e000/0xf62499ab9272730b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.770759:0:13878:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e000/0xf62499ab9272730b lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1785220585.770774:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].1246, 440 off1071400 00000004:00080000:1.0:1785220585.770780:0:6516:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff960d8d896900 with version 2431 00000004:00080000:1.0:1785220585.770791:0:7470:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff960ea686e240 version 2431 rpc_version 2431 00000020:00100000:1.0:1785220585.770814:0:7470:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1785220585.770822:0:7470:0:(client.c:807:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff960d8add6300 x1871938510812544/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1144/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1785220585.770846:0:7470:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add6300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510812544:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1785220585.770881:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1785220585.770884:0:7470:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220585.770887:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 2/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.770894:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ba15600 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:0.0:1785220585.770899:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.770906:0:7470:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896f00 going to sleep for 11 seconds 00000004:00010000:0.0:1785220585.770908:0:6514:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.770917:0:13876:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510812544 00000100:00100000:1.0:1785220585.770920:0:13876:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510812544 00010000:00010000:0.0:1785220585.770930:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 2/0,0 mode: --/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.770936:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1785220585.770941:0:13876:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12559 00000100:00100000:1.0:1785220585.770948:0:13876:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960ec1429850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0001-mdtlov_UUID+9:7470:x1871938510812544:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1785220585.770954:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 1/0,0 mode: --/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.770962:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ba15600/0xf62499ab92727319 lrc: 0/0,0 mode: --/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab92727327 expref: -99 pid: 13878 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.770976:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.770978:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.770980:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220585.771000:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1785220585.771009:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1871938510812672:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.771037:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.771072:0:13876:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960ec1429850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0001-mdtlov_UUID+10:7470:x1871938510812544:12345-0@lo:1000:osp_up0-1.0 Request processed in 124us (186us total) trans 4295000086 rc 0/0 00000100:00100000:1.0:1785220585.771079:0:13876:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12559 00000100:00100000:0.0:1785220585.771424:0:7470:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add6300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510812544:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1785220585.771492:0:13878:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e000/0xf62499ab9272730b lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771498:0:13878:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d9673e000 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.771501:0:13878:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e000/0xf62499ab9272730b lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771508:0:13878:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e000/0xf62499ab9272730b lrc: 4/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771518:0:13878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xd6e9df9d (ffff960d9902d600) 00010000:00010000:2.0:1785220585.771521:0:13878:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.771524:0:13878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771535:0:13878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771543:0:13878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 4/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771551:0:13878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771558:0:13878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771564:0:13878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 4/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771599:0:13878:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e000/0xf62499ab9272730b lrc: 2/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771606:0:13878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9673e000/0xf62499ab9272730b lrc: 0/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771620:0:13878:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771626:0:13878:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771637:0:13878:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771643:0:13878:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771651:0:13878:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94c7f200 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.771653:0:13878:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771659:0:13878:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40218400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771668:0:13878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:2.0:1785220585.771670:0:13878:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.771673:0:13878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771682:0:13878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771699:0:13878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 4/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771707:0:13878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adea00/0xf62499ab92794c30 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771714:0:13878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adea00/0xf62499ab92794c30 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771720:0:13878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adea00/0xf62499ab92794c30 lrc: 4/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771740:0:13878:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x44a19400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771747:0:13878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94c7f200/0xf62499ab927272e8 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x44a19400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771830:0:13878:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d95ba6000/0xf62499ab92727304 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771838:0:13878:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d95ba6000/0xf62499ab92727304 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771845:0:13878:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d95ba6000/0xf62499ab92727304 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771851:0:13878:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d95ba6000 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.771854:0:13878:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d95ba6000/0xf62499ab92727304 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771861:0:13878:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d95ba6000/0xf62499ab92727304 lrc: 4/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771869:0:13878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x84832a57 (ffff960d9902d400) 00010000:00010000:2.0:1785220585.771872:0:13878:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.771875:0:13878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771882:0:13878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771888:0:13878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 4/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771905:0:13878:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d95ba6000/0xf62499ab92727304 lrc: 2/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.771912:0:13878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d95ba6000/0xf62499ab92727304 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 13878 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.771978:0:13878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d984e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_008:5644b1e7-55e7-4421-a7e4-714e45fd656f+484:53574:x1871938507167488:12345-192.168.201.51@tcp:36:mrename.0 Request processed in 100046462us (100059096us total) trans 0 rc -39/-39 00000100:00100000:2.0:1785220585.771987:0:13878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 2875 00010000:00010000:2.0:1785220585.772011:0:8450:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772019:0:8450:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772220:0:8450:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5d3e lrc: 3/1,0 mode: --/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8450 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772232:0:8450:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5d3e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772240:0:8450:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5d3e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1785220585.772349:0:8450:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x9b0:0x0]: returning size 4096 00000004:00200000:2.0:1785220585.772354:0:8450:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1785220585.772374:0:8450:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5d3e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772384:0:8450:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8400/0xf62499ab927274d2 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772390:0:8450:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8400/0xf62499ab927274d2 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772396:0:8450:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8400/0xf62499ab927274d2 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772401:0:8450:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8d2f8400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.772404:0:8450:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8400/0xf62499ab927274d2 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772410:0:8450:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8400/0xf62499ab927274d2 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40218400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772419:0:8450:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:2.0:1785220585.772422:0:8450:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.772425:0:8450:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8400/0xf62499ab927274d2 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772431:0:8450:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8400/0xf62499ab927274d2 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772440:0:8450:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772445:0:8450:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772451:0:8450:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772455:0:8450:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8d2f8600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.772459:0:8450:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772464:0:8450:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772470:0:8450:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xd6e9df9d (ffff960d9902d600) 00010000:00010000:2.0:1785220585.772472:0:8450:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.772475:0:8450:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772480:0:8450:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f8600/0xf62499ab927274d9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8450 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.772504:0:8450:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8d2f9e00/0xf62499ab927274cb lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5cb4a expref: 14 pid: 8450 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00020000:2.0:1785220585.772518:0:8450:0:(ldlm_lockd.c:1453:ldlm_handle_enqueue()) ### lock on destroyed export 00000000cf022c12 ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5d3e lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5cb4a expref: 13 pid: 8450 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.774924:0:13812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.774942:0:13812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.775043:0:13812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3400/0xf62499ab927a5d45 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220585.776452:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510812672 00000100:00100000:3.0:1785220585.776465:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510812672 00000100:00100000:3.0:1785220585.776537:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14243 00000100:00100000:3.0:1785220585.776547:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:lustre-MDT0001-mdtlov_UUID+10:3652:x1871938510812672:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1785220585.776565:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220585.776595:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d9366fa00/0xf62499ab92727327 lrc: 0/0,0 mode: --/EX res: [0x200000404:0x806:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab92727319 expref: 10 pid: 13861 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.776617:0:13923:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x806:0x0].0x0 (ffff960d9902db00) 00010000:00010000:3.0:1785220585.776627:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220585.776714:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:lustre-MDT0001-mdtlov_UUID+9:3652:x1871938510812672:12345-0@lo:103:ldlm_bl.0 Request processed in 168us (5676us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.776726:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14243 00000100:00100000:3.0:1785220585.776848:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1871938510812672:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1785220585.776893:0:12310:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.776902:0:12310:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1785220585.776954:0:12310:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1785220585.776959:0:12310:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1785220585.776985:0:12310:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 12310 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.777013:0:12310:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195600/0xf62499ab927945b5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcbd expref: 14 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00020000:3.0:1785220585.777033:0:12310:0:(ldlm_lockd.c:1453:ldlm_handle_enqueue()) ### lock on destroyed export 00000000cf022c12 ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcbd expref: 13 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.777044:0:12310:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=-107) ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcbd expref: 13 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.777056:0:12310:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.777058:0:12310:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.777061:0:12310:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.777064:0:12310:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.777067:0:12310:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d92195000/0xf62499ab927945bc lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcbd expref: 13 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.777082:0:12310:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d92195000, rc -107) 00000100:00100000:3.0:1785220585.777836:0:12310:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9adb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+12:57310:x1871938518557568:12345-192.168.201.51@tcp:101:df.0 Request processed in 18613302us (18616105us total) trans 0 rc -107/-107 00000100:00100000:3.0:1785220585.777845:0:12310:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91782 00010000:00010000:3.0:1785220585.777880:0:13850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adea00/0xf62499ab92794c30 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.777889:0:13850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adea00/0xf62499ab92794c30 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1785220585.777918:0:13850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1785220585.777921:0:13850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1785220585.777947:0:13850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adf800/0xf62499ab92794c29 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fccb expref: 484 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.777963:0:13850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adea00/0xf62499ab92794c30 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fccb expref: 483 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.777971:0:13850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.777974:0:13850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1785220585.777977:0:13850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94adea00, rc 0) 00000100:00100000:3.0:1785220585.777998:0:13850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_019:5644b1e7-55e7-4421-a7e4-714e45fd656f+483:57312:x1871938518587264:12345-192.168.201.51@tcp:101:df.0 Request processed in 18269133us (18269221us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.778004:0:13850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91785 00010000:00010000:1.0:1785220585.778508:0:13860:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778518:0:13860:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778576:0:13860:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afea00/0xf62499ab92727382 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778588:0:13860:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afea00/0xf62499ab92727382 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778596:0:13860:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afea00/0xf62499ab92727382 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778602:0:13860:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97afea00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.778606:0:13860:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afea00/0xf62499ab92727382 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778613:0:13860:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afea00/0xf62499ab92727382 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778625:0:13860:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220585.778627:0:13860:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.778631:0:13860:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afea00/0xf62499ab92727382 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778637:0:13860:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afea00/0xf62499ab92727382 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778647:0:13860:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778653:0:13860:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778660:0:13860:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778665:0:13860:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d97afe800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.778668:0:13860:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778674:0:13860:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778682:0:13860:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x84832a57 (ffff960d9902d400) 00010000:00010000:1.0:1785220585.778684:0:13860:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.778703:0:13860:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778710:0:13860:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afe800/0xf62499ab92727389 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13860 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778756:0:13860:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-66, rc=-66) ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afec00/0xf62499ab9272737b lrc: 1/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ca6a expref: 11 pid: 13860 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778764:0:13860:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220585.778766:0:13860:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220585.778770:0:13860:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97afec00/0xf62499ab9272737b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ca6a expref: 11 pid: 13860 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.778780:0:13860:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d97afec00, rc -66) 00000100:00100000:1.0:1785220585.778828:0:13860:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d979a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+10:2407:x1871938507169280:12345-192.168.201.51@tcp:63:ll_sa.0 Request processed in 100014546us (100020344us total) trans 0 rc -66/-66 00000100:00100000:1.0:1785220585.778838:0:13860:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 10878 00000100:02000400:1.0:1785220585.778863:0:13860:0:(service.c:2915:ptlrpc_watchdog_update()) ll_ost_out00_00: service thread pid 13860 completed after 100.014s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources). 00010000:00010000:3.0:1785220585.778938:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220585.812039:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.812118:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.812148:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.812182:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.812206:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.812248:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520015872 00000100:00100000:3.0:1785220585.812259:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520015872 00000100:00100000:3.0:1785220585.812313:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520015744 00000100:00100000:3.0:1785220585.812317:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520015744 00000100:00100000:3.0:1785220585.812335:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520015616 00000100:00100000:3.0:1785220585.812338:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520015616 00000100:00100000:3.0:1785220585.812357:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 14244 00000100:00100000:3.0:1785220585.812366:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+483:2407:x1871938520015872:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220585.812381:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 16 locks, starting at 0 00010000:00010000:3.0:1785220585.812407:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2c6400/0xf62499ab924dab71 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x19a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a76e13 expref: 483 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812428:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19a:0x0].0x0 (ffff960d8c94b000) 00010000:00010000:3.0:1785220585.812446:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a439800/0xf62499ab924be4fd lrc: 0/0,0 mode: --/CR res: [0x240000402:0xf3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a6d8d5 expref: 482 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812458:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf3:0x0].0x0 (ffff960d8a4a3f00) 00010000:00010000:3.0:1785220585.812471:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89f4a000/0xf62499ab924d9ba3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x192:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a768e8 expref: 481 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812480:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x192:0x0].0x0 (ffff960d8aee6600) 00010000:00010000:3.0:1785220585.812491:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d89eaf600/0xf62499ab924d861e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x188:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a76282 expref: 480 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812500:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x188:0x0].0x0 (ffff960d89fe5000) 00010000:00010000:3.0:1785220585.812512:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfb8cc00/0xf62499ab924d730f lrc: 0/0,0 mode: --/CR res: [0x240000402:0x181:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75cd9 expref: 479 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812521:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x181:0x0].0x0 (ffff960ea4cd7500) 00010000:00010000:3.0:1785220585.812530:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a2cc200/0xf62499ab924d7ca8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x19b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75f8e expref: 478 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812539:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x19b:0x0].0x0 (ffff960d87ba5600) 00010000:00010000:3.0:1785220585.812549:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc33e00/0xf62499ab924d79de lrc: 0/0,0 mode: --/PR res: [0x240000402:0x113:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75ec3 expref: 477 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812558:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x113:0x0].0x0 (ffff960ea4cd7a00) 00010000:00010000:3.0:1785220585.812564:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfbe0e00/0xf62499ab924c301a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x112:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a6f323 expref: 476 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812573:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x112:0x0].0x0 (ffff960d87ba5500) 00010000:00010000:3.0:1785220585.812582:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8cc3dc00/0xf62499ab924d6efc lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75b97 expref: 475 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812591:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffff960d8bc0ec00) 00010000:00010000:3.0:1785220585.812601:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c5df200/0xf62499ab924d5dde lrc: 0/0,0 mode: --/CR res: [0x240000402:0x177:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a75657 expref: 474 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812609:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x177:0x0].0x0 (ffff960d86cfb600) 00010000:00010000:3.0:1785220585.812619:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8c590c00/0xf62499ab924d44a1 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x16b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a74e5b expref: 473 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812634:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffff960d8c1ede00) 00010000:00010000:3.0:1785220585.812643:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ec20cde00/0xf62499ab924d399c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x164:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a74af0 expref: 472 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812654:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x164:0x0].0x0 (ffff960d87b81300) 00010000:00010000:3.0:1785220585.812664:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8bc3b600/0xf62499ab924cd8c9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xfc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a72c57 expref: 471 pid: 13857 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812676:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8a276200/0xf62499ab924ca5fb lrc: 0/0,0 mode: --/CR res: [0x240000402:0xfc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a71b78 expref: 470 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812685:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xfc:0x0].0x0 (ffff960d8a12df00) 00010000:00010000:3.0:1785220585.812710:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960ebfbe0400/0xf62499ab924cb0d6 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x13e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a71f76 expref: 469 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812719:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x13e:0x0].0x0 (ffff960d89e82200) 00010000:00010000:3.0:1785220585.812728:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d88434800/0xf62499ab924c9c69 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x13a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7187d expref: 468 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812735:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x13a:0x0].0x0 (ffff960d8a12cb00) 00010000:00010000:3.0:1785220585.812741:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220585.812798:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5ed880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+467:2407:x1871938520015872:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 433us (768us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.812804:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 14244 00000100:00100000:3.0:1785220585.812823:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 14245 00000100:00100000:3.0:1785220585.812830:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97985880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+704:2407:x1871938520015744:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220585.812837:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 17 locks, starting at 0 00010000:00010000:3.0:1785220585.812847:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d88fdc600/0xf62499ab92494757 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x54:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a5f5a9 expref: 704 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812857:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x54:0x0].0x0 (ffff960d87b82a00) 00010000:00010000:3.0:1785220585.812869:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8b6cac00/0xf62499ab924c786d lrc: 0/0,0 mode: --/CR res: [0x200000403:0x155:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a70cd7 expref: 703 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812880:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x155:0x0].0x0 (ffff960d8a4a3c00) 00010000:00010000:3.0:1785220585.812887:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8c21aa00/0xf62499ab924d26be lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1ae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a74563 expref: 702 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812896:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1ae:0x0].0x0 (ffff960d8c1eda00) 00010000:00010000:3.0:1785220585.812905:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d89f48600/0xf62499ab924bee88 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x147:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a6dc39 expref: 701 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812915:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x147:0x0].0x0 (ffff960d8bde1400) 00010000:00010000:3.0:1785220585.812925:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d88f36600/0xf62499ab924d188d lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1f6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a73e08 expref: 700 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812933:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1f6:0x0].0x0 (ffff960e829cac00) 00010000:00010000:3.0:1785220585.812943:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8b1e0a00/0xf62499ab924d0bc1 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a73b30 expref: 699 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812951:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x46:0x0].0x0 (ffff960eba8e7e00) 00010000:00010000:3.0:1785220585.812959:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8b6e1400/0xf62499ab924cf007 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a73396 expref: 698 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812970:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a1:0x0].0x0 (ffff960d8bde2200) 00010000:00010000:3.0:1785220585.812979:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8c7aaa00/0xf62499ab924cde6b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x192:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a72e64 expref: 697 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.812988:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x192:0x0].0x0 (ffff960d8c94bb00) 00010000:00010000:3.0:1785220585.812997:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960ebfbe1e00/0xf62499ab924cdd7d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x18e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a72e2c expref: 696 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813004:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18e:0x0].0x0 (ffff960d8c218f00) 00010000:00010000:3.0:1785220585.813013:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8bc30e00/0xf62499ab924cd542 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x18f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a72ac1 expref: 695 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813022:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18f:0x0].0x0 (ffff960d8c218300) 00010000:00010000:3.0:1785220585.813030:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d87b7de00/0xf62499ab924cc0c0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x189:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a723cf expref: 694 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813039:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x189:0x0].0x0 (ffff960eba8e7700) 00010000:00010000:3.0:1785220585.813050:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8c21bc00/0xf62499ab924cbfb6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x188:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a72397 expref: 693 pid: 13809 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813059:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x188:0x0].0x0 (ffff960d86cfbb00) 00010000:00010000:3.0:1785220585.813067:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960ec20cc400/0xf62499ab924cb735 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x46:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7216e expref: 692 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813079:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x46:0x0].0x0 (ffff960eba8e7e00) 00010000:00010000:3.0:1785220585.813108:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d89a5ae00/0xf62499ab924c9bf9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x17a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a71868 expref: 691 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813118:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17a:0x0].0x0 (ffff960d8a12cc00) 00010000:00010000:3.0:1785220585.813128:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960ebfb8c200/0xf62499ab924aad99 lrc: 0/0,0 mode: --/CR res: [0x200000403:0xdd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a66fe3 expref: 690 pid: 13811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813135:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0xdd:0x0].0x0 (ffff960d8bc0e400) 00010000:00010000:3.0:1785220585.813144:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d89072400/0xf62499ab9249d24d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a6266a expref: 689 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813153:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff960eb7389f00) 00010000:00010000:3.0:1785220585.813163:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8a340a00/0xf62499ab924c86f9 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x155:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1a7118b expref: 688 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813171:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x155:0x0].0x0 (ffff960d8a4a3c00) 00010000:00010000:3.0:1785220585.813176:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220585.813194:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97985880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+687:2407:x1871938520015744:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 365us (1108us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.813200:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 14245 00000100:00100000:3.0:1785220585.813213:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 14246 00000100:00100000:3.0:1785220585.813218:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eba9ae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0+-99:2406:x1871938520015616:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00020000:3.0:1785220585.813223:0:6511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 192.168.201.51@tcp arrived at 1785220585 with bad export cookie 17736470194664537147 00000100:00100000:3.0:1785220585.813258:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eba9ae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:0+-99:2406:x1871938520015616:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 39us (1076us total) trans 0 rc -107/-107 00000100:00100000:3.0:1785220585.813264:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 14246 00000100:00100000:3.0:1785220585.813309:0:13807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520016000 00000100:00100000:3.0:1785220585.813313:0:13807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520016000 00000100:00100000:3.0:1785220585.813348:0:13807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520016128 00000100:00100000:3.0:1785220585.813351:0:13807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520016128 00000100:00100000:3.0:1785220585.813372:0:13807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91786 00000100:00100000:3.0:1785220585.813379:0:13807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d313b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:5644b1e7-55e7-4421-a7e4-714e45fd656f+467:53600:x1871938520016000:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:3.0:1785220585.813402:0:13807:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220585.813417:0:13807:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a5d4c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fcd2 expref: -99 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813510:0:13807:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 13807 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813524:0:13807:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813534:0:13807:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813547:0:13807:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13807 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813553:0:13807:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813560:0:13807:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813620:0:13807:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a5d61 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13807 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813627:0:13807:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a5d61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813633:0:13807:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a5d61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1785220585.813653:0:13807:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a5d61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813666:0:13807:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813672:0:13807:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813678:0:13807:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813686:0:13807:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813706:0:13807:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69000 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.813710:0:13807:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813718:0:13807:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813727:0:13807:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.813731:0:13807:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.813735:0:13807:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813742:0:13807:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69000/0xf62499ab927a5d53 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813755:0:13807:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813761:0:13807:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813767:0:13807:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813773:0:13807:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.813776:0:13807:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813782:0:13807:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813790:0:13807:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x18618557 (ffff960d8e268500) 00010000:00010000:3.0:1785220585.813792:0:13807:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.813795:0:13807:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813802:0:13807:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a5d5a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813830:0:13807:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a5d4c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcd2 expref: 469 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813848:0:13807:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a5d61 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcd2 expref: 468 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.813856:0:13807:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d8e268b00) 00010000:00010000:3.0:1785220585.813859:0:13807:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.813862:0:13807:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea68800, rc 0) 00000100:00100000:3.0:1785220585.813887:0:13807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d313b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:5644b1e7-55e7-4421-a7e4-714e45fd656f+468:53600:x1871938520016000:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 508us (1738us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.813892:0:13807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91786 00000100:00100000:3.0:1785220585.813909:0:13807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91787 00000100:00100000:3.0:1785220585.813916:0:13807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d310a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:5644b1e7-55e7-4421-a7e4-714e45fd656f+687:57312:x1871938520016128:12345-192.168.201.51@tcp:41:df.0 00000004:00000024:3.0:1785220585.813950:0:13807:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 7795449856 avail 7331946496 00000100:00100000:3.0:1785220585.813978:0:13807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d310a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:5644b1e7-55e7-4421-a7e4-714e45fd656f+687:57312:x1871938520016128:12345-192.168.201.51@tcp:41:df.0 Request processed in 63us (1770us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.813985:0:13807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91787 00000100:00080000:3.0:1785220585.814015:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1785220585.814019:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000004:00080000:3.0:1785220585.814406:0:6516:0:(mdd_object.c:4016:mdd_close()) Object [0x240000403:0xa20:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1785220585.814525:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].1243, 440 off892488 00000040:00080000:3.0:1785220585.814679:0:6516:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].1247, 440 off1071840 00000004:00080000:3.0:1785220585.814685:0:6516:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff960d8d38c300 with version 2432 00000004:00080000:3.0:1785220585.814713:0:7470:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff960ea686e240 version 2432 rpc_version 2432 00000020:00100000:3.0:1785220585.814740:0:7470:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1785220585.814748:0:7470:0:(client.c:807:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff960d8d312300 x1871938510812928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1144/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1785220585.814774:0:7470:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510812928:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1785220585.814874:0:7470:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220585.814913:0:7470:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38cd80 going to sleep for 11 seconds 00010000:00010000:2.0:1785220585.816026:0:8450:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=-107) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5d3e lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5cb4a expref: 9 pid: 8450 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.816513:0:13812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3400/0xf62499ab927a5d45 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.816528:0:13812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3400/0xf62499ab927a5d45 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.816836:0:8450:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9b0:0x0].0x0 (ffff960d934bd300) 00010000:00010000:2.0:1785220585.816840:0:8450:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1785220585.816844:0:8450:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9b0:0x0].0x0 (ffff960d934bd300) 00010000:00010000:2.0:1785220585.816846:0:8450:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1785220585.816851:0:8450:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5d3e lrc: 0/0,0 mode: --/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5cb4a expref: 9 pid: 8450 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.816871:0:8450:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b92200, rc -107) 00000100:00100000:0.0:1785220585.817176:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510812928 00000100:00100000:0.0:1785220585.817184:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510812928 00000100:00100000:0.0:1785220585.817231:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12560 00000100:00100000:0.0:1785220585.817241:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7470:x1871938510812928:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1785220585.817469:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d311880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7470:x1871938510812928:12345-0@lo:1000:osp_up0-1.0 Request processed in 229us (2595us total) trans 4295000087 rc 0/0 00000100:00100000:0.0:1785220585.817476:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12560 00000100:00100000:0.0:1785220585.817537:0:7470:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8d312300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510812928:0@lo:1000:osp_up0-1.0 00000001:00000004:0.0:1785220585.817604:0:6516:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-MDT0001_UUID: mount_count is 1, last_transno is 4294992691 00000020:00080000:0.0:1785220585.817653:0:6516:0:(genops.c:1672:class_fail_export()) disconnected export ffff960ebd9da800/45383443-d1a2-4f43-8eb5-d11e3b9f90c7 00010000:00010000:0.0:1785220585.817661:0:6516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d97980600/0xf62499ab927272e1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5ca63 expref: 5 pid: 13860 timeout: 1057 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1785220585.817867:0:13812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x9b0:0x0]: returning size 4096 00000004:00200000:3.0:1785220585.817872:0:13812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1785220585.817894:0:13812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3400/0xf62499ab927a5d45 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817908:0:13812:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817914:0:13812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817921:0:13812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817927:0:13812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817934:0:13812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8f40c000 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.817938:0:13812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817945:0:13812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817955:0:13812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.817957:0:13812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.817960:0:13812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817970:0:13812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8f40c000/0xf62499ab927295b7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817983:0:13812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817989:0:13812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.817995:0:13812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818000:0:13812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d9491c200 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.818003:0:13812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818009:0:13812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818015:0:13812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xd6e9df9d (ffff960d9902d600) 00010000:00010000:3.0:1785220585.818017:0:13812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.818020:0:13812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818025:0:13812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9491c200/0xf62499ab927295be lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd6e9df9d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13812 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818056:0:13812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ef82400/0xf62499ab927295b0 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d9c8 expref: 5 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00020000:3.0:1785220585.818069:0:13812:0:(ldlm_lockd.c:1453:ldlm_handle_enqueue()) ### lock on destroyed export 00000000cf022c12 ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3400/0xf62499ab927a5d45 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d9c8 expref: 4 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818079:0:13812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=-107) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3400/0xf62499ab927a5d45 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d9c8 expref: 4 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818165:0:13812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9b0:0x0].0x0 (ffff960d934bd300) 00010000:00010000:3.0:1785220585.818168:0:13812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.818170:0:13812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x9b0:0x0].0x0 (ffff960d934bd300) 00010000:00010000:3.0:1785220585.818171:0:13812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.818175:0:13812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3400/0xf62499ab927a5d45 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x9b0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5d9c8 expref: 4 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.818186:0:13812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3400, rc -107) 00000100:00100000:2.0:1785220585.818268:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d986ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+3:2407:x1871938507177344:12345-192.168.201.51@tcp:63:ll_sa.0 Request processed in 100019531us (100026263us total) trans 0 rc -107/-107 00000100:00100000:2.0:1785220585.818278:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 10881 00000100:02000400:2.0:1785220585.818502:0:8450:0:(service.c:2915:ptlrpc_watchdog_update()) ll_ost_out00_00: service thread pid 8450 completed after 100.019s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources). 00000100:00100000:3.0:1785220585.818633:0:13812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+2:53759:x1871938507324800:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 98860515us (98867445us total) trans 0 rc -107/-107 00000100:00100000:3.0:1785220585.818641:0:13812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 83422 00000100:00100000:3.0:1785220585.824520:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220585.824593:0:13805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520016256 00000100:00100000:0.0:1785220585.824602:0:13805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520016256 00000100:00100000:0.0:1785220585.824669:0:13805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91788 00000100:00100000:0.0:1785220585.824680:0:13805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8d313100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5644b1e7-55e7-4421-a7e4-714e45fd656f+687:53600:x1871938520016256:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:0.0:1785220585.824712:0:13805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1785220585.824736:0:13805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3c00/0xf62499ab927a5d68 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fcd9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.824926:0:13805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3200/0xf62499ab927a5d6f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.824942:0:13805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3200/0xf62499ab927a5d6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.824954:0:13805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3200/0xf62499ab927a5d6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1785220585.825001:0:13805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb05:0x0]: returning size 4096 00000004:00200000:0.0:1785220585.825005:0:13805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:0.0:1785220585.825041:0:13805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3c00/0xf62499ab927a5d68 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcd9 expref: 689 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.825060:0:13805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3200/0xf62499ab927a5d6f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcd9 expref: 688 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.825068:0:13805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d94c6c900) 00010000:00010000:0.0:1785220585.825071:0:13805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1785220585.825074:0:13805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3200, rc 0) 00000100:00100000:0.0:1785220585.825891:0:13805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8d313100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5644b1e7-55e7-4421-a7e4-714e45fd656f+688:53600:x1871938520016256:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 1210us (1377us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.825899:0:13805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91788 00000100:00100000:3.0:1785220585.829079:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.829236:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.829260:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.829280:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.829304:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520016384 00000100:00100000:3.0:1785220585.829313:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520016384 00000100:00100000:3.0:1785220585.829363:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520016640 00000100:00100000:3.0:1785220585.829366:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520016640 00000100:00100000:3.0:1785220585.829382:0:13853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520016512 00000100:00100000:3.0:1785220585.829384:0:13853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520016512 00000100:00100000:3.0:1785220585.829401:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 14247 00000100:00100000:3.0:1785220585.829410:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c83b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+470:2404:x1871938520016384:12345-192.168.201.51@tcp:103:mrename.0 00010000:00010000:3.0:1785220585.829426:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220585.829449:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94adea00/0xf62499ab92794c30 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fccb expref: 470 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.829467:0:13853:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.829473:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220585.829520:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c83b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+469:2404:x1871938520016384:12345-192.168.201.51@tcp:103:mrename.0 Request processed in 110us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.829527:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 14247 00000100:00100000:3.0:1785220585.829546:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 14248 00000100:00100000:3.0:1785220585.829553:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c83800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+468:2404:x1871938520016640:12345-192.168.201.51@tcp:103:mrename.0 00010000:00010000:3.0:1785220585.829561:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220585.829573:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9852f800/0xf62499ab927269b1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5c6c0 expref: 468 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.829584:0:13853:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xbcf:0x0].0x0 (ffff960d98bd4600) 00010000:00010000:3.0:1785220585.829594:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220585.829612:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c83800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+467:2404:x1871938520016640:12345-192.168.201.51@tcp:103:mrename.0 Request processed in 60us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.829618:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 14248 00000100:00100000:3.0:1785220585.829632:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 14249 00000100:00100000:3.0:1785220585.829638:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c80380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+466:2404:x1871938520016512:12345-192.168.201.51@tcp:103:mrename.0 00010000:00010000:3.0:1785220585.829645:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220585.829656:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a5d61 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcd2 expref: 466 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.829675:0:13853:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d8e268b00) 00010000:00010000:3.0:1785220585.829683:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220585.829711:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c80380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+465:2404:x1871938520016512:12345-192.168.201.51@tcp:103:mrename.0 Request processed in 73us (429us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.829717:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 14249 00000100:00100000:0.0:1785220585.829898:0:13858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520016896 00000100:00100000:0.0:1785220585.829903:0:13858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520016896 00000100:00100000:0.0:1785220585.829955:0:13858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 2911 00000100:00100000:0.0:1785220585.829962:0:13858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c80a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+688:53600:x1871938520016896:12345-192.168.201.51@tcp:36:mrename.0 00010000:00010000:0.0:1785220585.830005:0:13858:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220585.830021:0:13858:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3200/0xf62499ab927a5d6f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcd9 expref: 688 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830033:0:13858:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d94c6c900) 00010000:00010000:0.0:1785220585.830036:0:13858:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220585.830186:0:13858:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830196:0:13858:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830210:0:13858:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220585.830231:0:13858:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813056:0@lo:101:mdt_io00_007.0 00000100:00100000:0.0:1785220585.830273:0:13858:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220585.830312:0:13858:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f840 going to sleep for 11 seconds 00000100:00100000:0.0:1785220585.830342:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813056 00000100:00100000:0.0:1785220585.830346:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813056 00000100:00100000:0.0:1785220585.830372:0:13815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12561 00000100:00100000:0.0:1785220585.830378:0:13815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:13858:x1871938510813056:12345-0@lo:101:mdt_io00_007.0 00010000:00010000:0.0:1785220585.830388:0:13815:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1785220585.830399:0:13815:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5d7d lrc: 2/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a5d76 expref: -99 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830420:0:13815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5d7d lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a5d76 expref: 6 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830432:0:13815:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5d7d lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a5d76 expref: 6 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830439:0:13815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:0.0:1785220585.830442:0:13815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.830444:0:13815:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3e00, rc 0) 00000100:00100000:0.0:1785220585.830472:0:13815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:13858:x1871938510813056:12345-0@lo:101:mdt_io00_007.0 Request processed in 94us (199us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.830478:0:13815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12561 00000100:00100000:0.0:1785220585.830511:0:13858:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813056:0@lo:101:mdt_io00_007.0 00010000:00010000:0.0:1785220585.830528:0:13858:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830539:0:13858:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220585.830555:0:13858:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00010000:00010000:0.0:1785220585.830574:0:13858:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830582:0:13858:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830588:0:13858:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1785220585.830599:0:13858:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813184:0@lo:101:mdt_io00_007.0 00000100:00100000:0.0:1785220585.830620:0:13858:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220585.830639:0:13858:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f840 going to sleep for 11 seconds 00000100:00100000:0.0:1785220585.830651:0:13914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813184 00000100:00100000:0.0:1785220585.830654:0:13914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813184 00000100:00100000:0.0:1785220585.830673:0:13914:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12562 00000100:00100000:0.0:1785220585.830678:0:13914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_005:lustre-MDT0000-mdtlov_UUID+6:13858:x1871938510813184:12345-0@lo:101:mdt_io00_007.0 00010000:00010000:0.0:1785220585.830685:0:13914:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1785220585.830707:0:13914:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3800/0xf62499ab927a5d8b lrc: 2/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a5d84 expref: -99 pid: 13914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830721:0:13914:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3800/0xf62499ab927a5d8b lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a5d84 expref: 7 pid: 13914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830736:0:13914:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3800/0xf62499ab927a5d8b lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a5d84 expref: 7 pid: 13914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830743:0:13914:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xea196397 (ffff960d959b4f00) 00010000:00010000:0.0:1785220585.830745:0:13914:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.830748:0:13914:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3800, rc 0) 00000100:00100000:0.0:1785220585.830773:0:13914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_005:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813184:12345-0@lo:101:mdt_io00_007.0 Request processed in 95us (152us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.830779:0:13914:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12562 00000100:00100000:0.0:1785220585.830799:0:13858:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813184:0@lo:101:mdt_io00_007.0 00010000:00010000:0.0:1785220585.830808:0:13858:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.830816:0:13858:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1785220585.830826:0:13858:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00000100:00100000:0.0:1785220585.830857:0:13858:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813312:0@lo:1000:mdt_io00_007.0 00000100:00100000:0.0:1785220585.830880:0:13858:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220585.830894:0:13858:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:0.0:1785220585.830907:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813312 00000100:00100000:0.0:1785220585.830910:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813312 00000100:00100000:0.0:1785220585.830932:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12563 00000100:00100000:0.0:1785220585.830938:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813312:12345-0@lo:1000:mdt_io00_007.0 00000100:00100000:0.0:1785220585.830993:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813312:12345-0@lo:1000:mdt_io00_007.0 Request processed in 55us (112us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.830999:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12563 00000100:00100000:0.0:1785220585.831021:0:13858:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813312:0@lo:1000:mdt_io00_007.0 00000100:00100000:0.0:1785220585.831055:0:13858:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813440:0@lo:1000:mdt_io00_007.0 00000100:00100000:0.0:1785220585.831077:0:13858:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.831619:0:13876:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813440 00000100:00100000:1.0:1785220585.831624:0:13876:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813440 00000100:00100000:1.0:1785220585.831655:0:13876:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12564 00000100:00100000:1.0:1785220585.831661:0:13876:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813440:12345-0@lo:1000:mdt_io00_007.0 00000100:00080000:3.0:1785220585.831831:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1785220585.833915:0:13876:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813440:12345-0@lo:1000:mdt_io00_007.0 Request processed in 2245us (2827us total) trans 0 rc 1/1 00000100:00100000:1.0:1785220585.833930:0:13876:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12564 00000100:00100000:1.0:1785220585.836005:0:13858:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d95b9f600 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.836054:0:13858:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813440:0@lo:1000:mdt_io00_007.0 00000100:00100000:1.0:1785220585.836195:0:13858:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813568:0@lo:1000:mdt_io00_007.0 00000100:00100000:1.0:1785220585.836257:0:13858:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.836284:0:13858:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.836308:0:13845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813568 00000100:00100000:1.0:1785220585.836314:0:13845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813568 00000100:00100000:1.0:1785220585.836364:0:13845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12565 00000100:00100000:1.0:1785220585.836373:0:13845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813568:12345-0@lo:1000:mdt_io00_007.0 00000100:00100000:1.0:1785220585.836464:0:13845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813568:12345-0@lo:1000:mdt_io00_007.0 Request processed in 92us (207us total) trans 0 rc 1/1 00000100:00100000:1.0:1785220585.836471:0:13845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12565 00000100:00100000:1.0:1785220585.836502:0:13858:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813568:0@lo:1000:mdt_io00_007.0 00010000:00010000:1.0:1785220585.836552:0:13858:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836563:0:13858:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836573:0:13858:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.836582:0:13858:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813696:0@lo:101:mdt_io00_007.0 00000100:00100000:1.0:1785220585.836603:0:13858:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.836616:0:13858:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.836626:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813696 00000100:00100000:1.0:1785220585.836629:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813696 00000100:00100000:1.0:1785220585.836647:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12566 00000100:00100000:1.0:1785220585.836652:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:13858:x1871938510813696:12345-0@lo:101:mdt_io00_007.0 00010000:00010000:1.0:1785220585.836660:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.836677:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5d99 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a5d92 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836714:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5d99 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a5d92 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836726:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5d99 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a5d92 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836734:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d8a305500) 00010000:00010000:1.0:1785220585.836736:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1785220585.836740:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98497c00, rc 0) 00000100:00100000:1.0:1785220585.836768:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:13858:x1871938510813696:12345-0@lo:101:mdt_io00_007.0 Request processed in 115us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.836775:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12566 00000100:00100000:1.0:1785220585.836803:0:13858:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813696:0@lo:101:mdt_io00_007.0 00010000:00010000:1.0:1785220585.836815:0:13858:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836824:0:13858:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836846:0:13858:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836855:0:13858:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836863:0:13858:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836883:0:13858:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836890:0:13858:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.836896:0:13858:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.836907:0:13858:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813824:0@lo:101:mdt_io00_007.0 00000100:00100000:1.0:1785220585.836927:0:13858:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.836942:0:13858:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896240 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.836953:0:13876:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813824 00000100:00100000:1.0:1785220585.836956:0:13876:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813824 00000100:00100000:1.0:1785220585.836978:0:13876:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12567 00000100:00100000:1.0:1785220585.836984:0:13876:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0000-mdtlov_UUID+8:13858:x1871938510813824:12345-0@lo:101:mdt_io00_007.0 00010000:00010000:1.0:1785220585.836991:0:13876:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.837001:0:13876:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5dae lrc: 2/0,0 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a5da7 expref: -99 pid: 13876 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837020:0:13876:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5dae lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a5da7 expref: 9 pid: 13876 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837030:0:13876:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5dae lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a5da7 expref: 9 pid: 13876 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837037:0:13876:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xbcf:0x0].0x0 (ffff960d8a305800) 00010000:00010000:1.0:1785220585.837039:0:13876:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1785220585.837041:0:13876:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496a00, rc 0) 00000100:00100000:1.0:1785220585.837068:0:13876:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_004:lustre-MDT0000-mdtlov_UUID+9:13858:x1871938510813824:12345-0@lo:101:mdt_io00_007.0 Request processed in 84us (139us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.837074:0:13876:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12567 00000100:00100000:1.0:1785220585.837120:0:13858:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_007:lustre-MDT0000-mdtlov_UUID:13858:1871938510813824:0@lo:101:mdt_io00_007.0 00010000:00010000:1.0:1785220585.837129:0:13858:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837137:0:13858:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837156:0:13858:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837163:0:13858:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837171:0:13858:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837197:0:13858:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837205:0:13858:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837212:0:13858:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837308:0:13858:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837316:0:13858:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837324:0:13858:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837330:0:13858:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.837334:0:13858:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837342:0:13858:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837351:0:13858:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d94c6c900) 00010000:00010000:1.0:1785220585.837354:0:13858:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.837358:0:13858:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837364:0:13858:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497800/0xf62499ab927a5da0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837377:0:13858:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837384:0:13858:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837391:0:13858:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837409:0:13858:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220585.837414:0:13858:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837422:0:13858:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.837428:0:13858:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842016:0:13858:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1785220585.842045:0:13858:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842058:0:13858:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842067:0:13858:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842078:0:13858:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.842097:0:13858:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842119:0:13858:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842140:0:13858:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xbcf:0x0].0x0 (ffff960d90056200) 00010000:00010000:1.0:1785220585.842143:0:13858:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1785220585.842149:0:13858:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842156:0:13858:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5db5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220585.842190:0:13858:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00010000:00010000:1.0:1785220585.842198:0:13858:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842207:0:13858:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.842214:0:13858:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.845841:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1785220585.845858:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.845872:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496c00 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:0.0:1785220585.845880:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1785220585.845889:0:6515:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.845934:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.845941:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1785220585.845976:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 1/0,0 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.845984:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496c00/0xf62499ab927a5d92 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d99 expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846005:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1785220585.846009:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846015:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497600 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:0.0:1785220585.846019:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1785220585.846026:0:6515:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846041:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846047:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1785220585.846053:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 1/0,0 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846060:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497600/0xf62499ab927a5da7 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5dae expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846072:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1785220585.846076:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846082:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3000 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:0.0:1785220585.846175:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1785220585.846185:0:6515:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846209:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 4/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846215:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1785220585.846231:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846240:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.846241:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.846243:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220585.846267:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510814080:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.846336:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220585.846388:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220585.846394:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510813952:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.846415:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220585.846438:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1785220585.846446:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510814208:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.846466:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220585.846492:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814080 00000100:00100000:0.0:1785220585.846499:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814080 00000100:00100000:0.0:1785220585.846541:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510813952 00000100:00100000:0.0:1785220585.846544:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510813952 00000100:00100000:0.0:1785220585.846559:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814208 00000100:00100000:0.0:1785220585.846562:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814208 00000100:00100000:0.0:1785220585.846582:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14250 00000100:00100000:0.0:1785220585.846589:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000-mdtlov_UUID+11:3655:x1871938510814080:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1785220585.846601:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220585.846619:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5dae lrc: 0/0,0 mode: --/EX res: [0x200000402:0xbcf:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab927a5da7 expref: 11 pid: 13876 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846633:0:14181:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xbcf:0x0].0x0 (ffff960d8a305800) 00010000:00010000:0.0:1785220585.846642:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220585.846675:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000-mdtlov_UUID+10:3655:x1871938510814080:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (338us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.846681:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14250 00000100:00100000:0.0:1785220585.846710:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14251 00000100:00100000:0.0:1785220585.846716:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000-mdtlov_UUID+9:3655:x1871938510813952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1785220585.846724:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220585.846734:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5d99 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab927a5d92 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846745:0:14181:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d8a305500) 00010000:00010000:0.0:1785220585.846751:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220585.846775:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000-mdtlov_UUID+8:3655:x1871938510813952:12345-0@lo:103:ldlm_bl.0 Request processed in 59us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.846780:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14251 00000100:00100000:0.0:1785220585.846792:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14252 00000100:00100000:0.0:1785220585.846798:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000-mdtlov_UUID+7:3653:x1871938510814208:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1785220585.846805:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220585.846815:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5d7d lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab927a5d76 expref: 7 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.846825:0:14181:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:0.0:1785220585.846828:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220585.846850:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000-mdtlov_UUID+6:3653:x1871938510814208:12345-0@lo:103:ldlm_bl.0 Request processed in 53us (382us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.846856:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14252 00000100:00100000:0.0:1785220585.846889:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510813952:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.846897:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510814080:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.846916:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a705f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510814208:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1785220585.846978:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220585.847828:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00010000:00010000:1.0:1785220585.848021:0:13858:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1785220585.848031:0:13858:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be3000/0xf62499ab927a5d76 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d7d expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220585.848063:0:13858:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00010000:00010000:1.0:1785220585.848069:0:13858:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848077:0:13858:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848805:0:13858:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848837:0:13858:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1785220585.848847:0:13858:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848854:0:13858:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848860:0:13858:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848866:0:13858:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497200 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.848870:0:13858:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848878:0:13858:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848892:0:13858:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff960d8a305700) 00010000:00010000:1.0:1785220585.848895:0:13858:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.848901:0:13858:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848907:0:13858:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497200/0xf62499ab927a5dbc lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13858 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848975:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220585.848980:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.848988:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2400 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:1.0:1785220585.848992:0:13859:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1785220585.849000:0:13859:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.849034:0:13859:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 2/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.849040:0:13859:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1785220585.849063:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.849071:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d94be2400/0xf62499ab927a5d84 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5d8b expref: -99 pid: 13858 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.849187:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220585.849189:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220585.849191:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220585.849206:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510814336:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1785220585.849261:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.849317:0:14235:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814336 00000100:00100000:1.0:1785220585.849323:0:14235:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814336 00000100:00100000:1.0:1785220585.849364:0:14235:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14253 00000100:00100000:1.0:1785220585.849371:0:14235:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac541850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:lustre-MDT0000-mdtlov_UUID+6:3653:x1871938510814336:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1785220585.849382:0:14235:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220585.849398:0:14235:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3800/0xf62499ab927a5d8b lrc: 0/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xea196397 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab927a5d84 expref: 6 pid: 13914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.849412:0:14235:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xea196397 (ffff960d959b4f00) 00010000:00010000:1.0:1785220585.849419:0:14235:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220585.849449:0:14235:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac541850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_012:lustre-MDT0000-mdtlov_UUID+5:3653:x1871938510814336:12345-0@lo:103:ldlm_bl.0 Request processed in 77us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.849456:0:14235:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14253 00000100:00100000:1.0:1785220585.849517:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8add7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510814336:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1785220585.849571:0:13858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c80a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+687:53600:x1871938520016896:12345-192.168.201.51@tcp:36:mrename.0 Request processed in 19609us (20499us total) trans 0 rc -39/-39 00000100:00100000:1.0:1785220585.849579:0:13858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 2911 00000100:00100000:0.0:1785220585.850289:0:13918:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520017024 00000100:00100000:0.0:1785220585.850297:0:13918:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520017024 00000100:00100000:0.0:1785220585.850326:0:13918:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91789 00000100:00100000:0.0:1785220585.850333:0:13918:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_023:0+-99:2403:x1871938520017024:12345-192.168.201.51@tcp:38:df.0 00010000:00080000:0.0:1785220585.850356:0:13918:0:(ldlm_lib.c:1445:target_handle_connect()) lustre-MDT0001: connection from 45383443-d1a2-4f43-8eb5-d11e3b9f90c7@192.168.201.51@tcp t4294968028 exp 0000000000000000 cur 1057 last 0 00000020:00000080:0.0:1785220585.850388:0:13918:0:(genops.c:1406:class_connect()) connect: client 45383443-d1a2-4f43-8eb5-d11e3b9f90c7, cookie 0xf62499ab927a5dc3 00000004:00100000:0.0:1785220585.850410:0:13918:0:(mdt_handler.c:7282:mdt_connect_internal()) lustre-MDT0001: cli supports cksum type f7, return f7 00000100:00080000:0.0:1785220585.850514:0:13918:0:(import.c:65:import_set_state_nolock()) ffff960d978b2000 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1785220585.850552:0:13918:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_023:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+6:2403:x1871938520017024:12345-192.168.201.51@tcp:38:df.0 Request processed in 218us (2728us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.850558:0:13918:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91789 00010000:00010000:3.0:1785220585.850669:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1785220585.863056:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220585.863242:0:13919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520017152 00000100:00100000:0.0:1785220585.863252:0:13919:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520017152 00000100:00100000:0.0:1785220585.863316:0:13919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91790 00000100:00100000:0.0:1785220585.863327:0:13919:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_024:5644b1e7-55e7-4421-a7e4-714e45fd656f+465:53612:x1871938520017152:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:0.0:1785220585.863348:0:13919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1785220585.863371:0:13919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3a00/0xf62499ab927a5dca lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fce0 expref: -99 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863478:0:13919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863493:0:13919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863504:0:13919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863519:0:13919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863526:0:13919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863532:0:13919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863613:0:13919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2e00/0xf62499ab927a5ddf lrc: 3/1,0 mode: --/PR res: [0x240000403:0x9db:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863621:0:13919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2e00/0xf62499ab927a5ddf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x9db:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863628:0:13919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2e00/0xf62499ab927a5ddf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x9db:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1785220585.863666:0:13919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x9db:0x0]: returning size 4096 00000004:00200000:0.0:1785220585.863671:0:13919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1785220585.863701:0:13919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2e00/0xf62499ab927a5ddf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x9db:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863719:0:13919:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863726:0:13919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863732:0:13919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863738:0:13919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863744:0:13919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2000 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.863748:0:13919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863755:0:13919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863776:0:13919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:0.0:1785220585.863779:0:13919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.863784:0:13919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863790:0:13919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5dd1 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863802:0:13919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863808:0:13919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863815:0:13919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863820:0:13919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.863823:0:13919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863829:0:13919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863836:0:13919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xcdd6ff3 (ffff960d959b4c00) 00010000:00010000:0.0:1785220585.863838:0:13919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.863841:0:13919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863847:0:13919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5dd8 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xcdd6ff3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863876:0:13919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3a00/0xf62499ab927a5dca lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fce0 expref: 467 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863888:0:13919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2e00/0xf62499ab927a5ddf lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x9db:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fce0 expref: 466 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.863895:0:13919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9db:0x0].0x0 (ffff960d959b4e00) 00010000:00010000:0.0:1785220585.863897:0:13919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1785220585.863899:0:13919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be2e00, rc 0) 00000100:00100000:0.0:1785220585.864201:0:13919:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_024:5644b1e7-55e7-4421-a7e4-714e45fd656f+466:53612:x1871938520017152:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 875us (1154us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.864212:0:13919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91790 00000100:00100000:3.0:1785220585.885588:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.885633:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.885672:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.885706:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220585.885915:0:13814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520017408 00000100:00100000:2.0:1785220585.885926:0:13814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520017408 00000100:00100000:2.0:1785220585.885976:0:13814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520017792 00000100:00100000:2.0:1785220585.885979:0:13814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520017792 00000100:00100000:2.0:1785220585.885997:0:13814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520017920 00000100:00100000:2.0:1785220585.885999:0:13814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520017920 00000100:00100000:2.0:1785220585.886010:0:13814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520018432 00000100:00100000:2.0:1785220585.886012:0:13814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520018432 00000100:00100000:2.0:1785220585.886027:0:13814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91791 00000100:00100000:2.0:1785220585.886036:0:13814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c81180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+469:53627:x1871938520017408:12345-192.168.201.51@tcp:101:touch.0 00010000:00010000:2.0:1785220585.886057:0:13814:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1785220585.886061:0:13814:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1785220585.886131:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2e00/0xf62499ab927a5ddf lrc: 0/0,0 mode: --/PR res: [0x240000403:0x9db:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fce0 expref: 469 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886150:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9db:0x0].0x0 (ffff960d959b4e00) 00010000:00010000:2.0:1785220585.886161:0:13814:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1785220585.886175:0:13814:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5de6 lrc: 2/0,0 mode: --/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fce7 expref: -99 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886311:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 3/0,1 mode: --/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886324:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 3/0,1 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886334:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 3/0,1 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886345:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92a00/0xf62499ab927a5df4 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x9db:0x0].0xf7a5b27a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886352:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92a00/0xf62499ab927a5df4 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x9db:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886357:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92a00/0xf62499ab927a5df4 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x9db:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1785220585.886655:0:13814:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x507a:0x0], index 1: 0 00000100:00100000:1.0:1785220585.886789:0:13826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91792 00000100:00100000:1.0:1785220585.886803:0:13826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c83b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_016:5644b1e7-55e7-4421-a7e4-714e45fd656f+470:53673:x1871938520017792:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:1.0:1785220585.886813:0:13826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1785220585.886824:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92000/0xf62499ab927a5dfb lrc: 3/1,0 mode: --/PR res: [0x240000402:0xace:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.886827:0:13826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497200/0xf62499ab927a5e02 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fcf5 expref: -99 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886842:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92000/0xf62499ab927a5dfb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xace:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886851:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92000/0xf62499ab927a5dfb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xace:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886880:0:13814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 4/0,1 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886887:0:13814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 4/0,1 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886894:0:13814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 3/0,0 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.886896:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886903:0:13814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 4/0,0 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.886911:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886913:0:13814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.886920:0:13814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 4/0,0 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.886925:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886929:0:13814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 5/0,0 mode: CW/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886943:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9db:0x0].0x0 (ffff960d9843b600) 00010000:00010000:1.0:1785220585.886944:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886948:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.886955:0:13814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 3/0,0 mode: --/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.886958:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886969:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93800/0xf62499ab927a5ded lrc: 0/0,0 mode: --/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.886972:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1785220585.886985:0:13814:0:(mdt_handler.c:4478:mdt_save_lock()) ### save lock request 00000000fb169748 reply state 00000000d6e49f98 transno 4294992692 ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92a00/0xf62499ab927a5df4 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x9db:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.886998:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92a00/0xf62499ab927a5df4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x9db:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887004:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9db:0x0].0xf7a5b27a (ffff960d9843b500) 00010000:00010000:2.0:1785220585.887007:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1785220585.887009:0:13814:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1785220585.887033:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497e00/0xf62499ab927a5e1e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x9b6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887036:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5de6 lrc: 0/0,0 mode: --/CW res: [0x240000403:0x9db:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fce7 expref: 472 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887050:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497e00/0xf62499ab927a5e1e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x9b6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887066:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497e00/0xf62499ab927a5e1e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x9b6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887074:0:13814:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92000/0xf62499ab927a5dfb lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xace:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fce7 expref: 471 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887082:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xace:0x0].0x0 (ffff960d9843b400) 00010000:00010000:2.0:1785220585.887122:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1785220585.887125:0:13814:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b92000, rc 0) 00000004:00200000:1.0:1785220585.887160:0:13826:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x9b6:0x0]: returning size 4096 00000004:00200000:1.0:1785220585.887166:0:13826:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1785220585.887179:0:13826:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497e00/0xf62499ab927a5e1e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x9b6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.887188:0:13814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c81180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+472:53627:x1871938520017408:12345-192.168.201.51@tcp:101:touch.0 Request processed in 1150us (1608us total) trans 4294992692 rc 0/0 00010000:00010000:1.0:1785220585.887191:0:13826:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.887201:0:13814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91791 00010000:00010000:1.0:1785220585.887210:0:13826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887216:0:13826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887223:0:13826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.887229:0:13814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91793 00010000:00010000:1.0:1785220585.887232:0:13826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.887238:0:13826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.887240:0:13814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c83800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+471:55377:x1871938520017920:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:1.0:1785220585.887250:0:13826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887254:0:13814:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.887263:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:2.0:1785220585.887266:0:13814:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92e00/0xf62499ab927a5e25 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fcfc expref: -99 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887267:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.887272:0:13826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887283:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496600/0xf62499ab927a5e10 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887294:0:13826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887301:0:13826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887307:0:13826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887312:0:13826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.887315:0:13826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887321:0:13826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887326:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887329:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x9b50d0cc (ffff960d8a305700) 00010000:00010000:1.0:1785220585.887334:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.887339:0:13826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887343:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887348:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5e17 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x9b50d0cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887356:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887371:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887376:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497200/0xf62499ab927a5e02 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcf5 expref: 473 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887397:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887403:0:13826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497e00/0xf62499ab927a5e1e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x9b6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcf5 expref: 472 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887409:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887415:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x9b6:0x0].0x0 (ffff960d8a305400) 00010000:00010000:1.0:1785220585.887420:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1785220585.887422:0:13826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98497e00, rc 0) 00010000:00010000:2.0:1785220585.887444:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92c00/0xf62499ab927a5e3a lrc: 3/1,0 mode: --/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.887449:0:13826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c83b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_016:5644b1e7-55e7-4421-a7e4-714e45fd656f+472:53673:x1871938520017792:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 650us (1814us total) trans 0 rc 0/0 00010000:00010000:2.0:1785220585.887458:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92c00/0xf62499ab927a5e3a lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887465:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92c00/0xf62499ab927a5e3a lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.887477:0:13826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91792 00000004:00010000:2.0:1785220585.887479:0:13814:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92c00/0xf62499ab927a5e3a lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887491:0:13814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.887497:0:13826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91794 00010000:00010000:2.0:1785220585.887500:0:13814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.887509:0:13826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d98c80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_016:5644b1e7-55e7-4421-a7e4-714e45fd656f+471:53889:x1871938520018432:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:2.0:1785220585.887510:0:13814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887520:0:13826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1785220585.887521:0:13814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887528:0:13814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93c00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.887530:0:13826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a5e41 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd0a expref: -99 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887532:0:13814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887550:0:13814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887559:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:2.0:1785220585.887561:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.887565:0:13814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887570:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93c00/0xf62499ab927a5e2c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887580:0:13814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887587:0:13814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887593:0:13814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887594:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887604:0:13814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93200 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.887609:0:13814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887610:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887622:0:13814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887626:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887634:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xbc0ca3a9 (ffff960d9843bb00) 00010000:00010000:2.0:1785220585.887638:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220585.887642:0:13814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887643:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887662:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887663:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93200/0xf62499ab927a5e33 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887676:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887705:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92e00/0xf62499ab927a5e25 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcfc expref: 473 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887716:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5e56 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887721:0:13814:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92c00/0xf62499ab927a5e3a lrc: 3/0,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcfc expref: 472 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887727:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5e56 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887733:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x7c5:0x0].0x0 (ffff960d9843b000) 00010000:00010000:2.0:1785220585.887738:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1785220585.887739:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5e56 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.887741:0:13814:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b92c00, rc 0) 00000004:00010000:1.0:1785220585.887760:0:13826:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5e56 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.887768:0:13814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c83800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+472:55377:x1871938520017920:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 531us (2095us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220585.887770:0:13826:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.887781:0:13814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91793 00010000:00010000:1.0:1785220585.887784:0:13826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887791:0:13826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887797:0:13826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887802:0:13826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496200 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.887805:0:13826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887810:0:13826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887818:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220585.887821:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.887823:0:13826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887829:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5e48 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887838:0:13826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887845:0:13826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887850:0:13826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887855:0:13826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.887859:0:13826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887866:0:13826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887873:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x18618557 (ffff960d8a305f00) 00010000:00010000:1.0:1785220585.887875:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.887877:0:13826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887883:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a5e4f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887903:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a5e41 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd0a expref: 472 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887913:0:13826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497c00/0xf62499ab927a5e56 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd0a expref: 471 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.887920:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d8a305300) 00010000:00010000:1.0:1785220585.887922:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1785220585.887924:0:13826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98497c00, rc 0) 00000100:00100000:2.0:1785220585.888450:0:13826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d98c80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_016:5644b1e7-55e7-4421-a7e4-714e45fd656f+471:53889:x1871938520018432:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 943us (2742us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220585.888458:0:13826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91794 00000100:00100000:0.0:1785220585.910908:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.911173:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.911352:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220585.911476:0:13823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520020096 00000100:00100000:0.0:1785220585.911491:0:13823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520020096 00000100:00100000:0.0:1785220585.911565:0:13823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 25170 00000100:00100000:0.0:1785220585.911576:0:13823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:53602:x1871938520020096:12345-192.168.201.51@tcp:35:rm.0 00000100:00100000:0.0:1785220585.911828:0:13823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_004:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:53602:x1871938520020096:12345-192.168.201.51@tcp:35:rm.0 Request processed in 254us (476us total) trans 4295000088 rc 0/0 00000100:00100000:0.0:1785220585.911836:0:13823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 25170 00000100:00100000:1.0:1785220585.913000:0:13871:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520017664 00000100:00100000:1.0:1785220585.913010:0:13871:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520017664 00000100:00100000:1.0:1785220585.913143:0:13871:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520018816 00000100:00100000:1.0:1785220585.913148:0:13871:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520018816 00000100:00080000:3.0:1785220585.913225:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1785220585.913430:0:13813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91795 00000100:00100000:0.0:1785220585.913441:0:13813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_014:5644b1e7-55e7-4421-a7e4-714e45fd656f+471:53670:x1871938520017664:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:0.0:1785220585.913593:0:13813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1785220585.913607:0:13808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91796 00010000:00010000:0.0:1785220585.913618:0:13813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5e5d lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fcee expref: -99 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.913619:0:13808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:53607:x1871938520018816:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:2.0:1785220585.913634:0:13808:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1785220585.913653:0:13808:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92e00/0xf62499ab927a5e64 lrc: 2/0,0 mode: --/CR res: [0x200000404:0x740:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd1f expref: -99 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913745:0:13813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913763:0:13813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913774:0:13813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913787:0:13813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913794:0:13813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913800:0:13813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913849:0:13813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3200/0xf62499ab927a5e79 lrc: 3/1,0 mode: --/PR res: [0x200000404:0x806:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913857:0:13813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3200/0xf62499ab927a5e79 lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913863:0:13813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3200/0xf62499ab927a5e79 lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913876:0:13813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913882:0:13813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913889:0:13813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913895:0:13813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913902:0:13813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2000 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.913906:0:13813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913912:0:13813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913922:0:13813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:0.0:1785220585.913926:0:13813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.913930:0:13813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913937:0:13813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2000/0xf62499ab927a5e6b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913950:0:13813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913956:0:13813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913962:0:13813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913967:0:13813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3e00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220585.913970:0:13808:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92e00/0xf62499ab927a5e64 lrc: 1/0,0 mode: --/CR res: [0x200000404:0x740:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd1f expref: 589 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913971:0:13813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.913988:0:13813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.914007:0:13808:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x740:0x0].0x0 (ffff960d9843bb00) 00010000:00010000:0.0:1785220585.914007:0:13813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x84832a57 (ffff960d959b4c00) 00010000:00010000:2.0:1785220585.914014:0:13808:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.914014:0:13813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.914019:0:13813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.914021:0:13808:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92e00/0xf62499ab927a5e64 lrc: 0/0,0 mode: --/CR res: [0x200000404:0x740:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd1f expref: 589 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.914031:0:13813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3e00/0xf62499ab927a5e72 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.914049:0:13808:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b92e00, rc 0) 00010000:00010000:0.0:1785220585.914075:0:13813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5e5d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcee expref: 473 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.914279:0:13813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3200/0xf62499ab927a5e79 lrc: 3/0,0 mode: PR/PR res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fcee expref: 472 pid: 13813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.914286:0:13813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x806:0x0].0x0 (ffff960d959b4900) 00010000:00010000:0.0:1785220585.914288:0:13813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1785220585.914290:0:13813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3200, rc 0) 00000100:00100000:0.0:1785220585.914326:0:13813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_014:5644b1e7-55e7-4421-a7e4-714e45fd656f+472:53670:x1871938520017664:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 888us (3433us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.914334:0:13813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91795 00000100:00100000:3.0:1785220585.914648:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.914678:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.914725:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520020224 00000100:00100000:3.0:1785220585.914730:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520020224 00000100:00100000:3.0:1785220585.914758:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520020352 00000100:00100000:3.0:1785220585.914761:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520020352 00000100:00100000:3.0:1785220585.914840:0:13921:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91797 00000100:00100000:3.0:1785220585.914849:0:13921:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:5644b1e7-55e7-4421-a7e4-714e45fd656f+688:55377:x1871938520020224:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:3.0:1785220585.914859:0:13921:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220585.914874:0:13921:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68a00/0xf62499ab927a5e80 lrc: 2/0,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd5e expref: -99 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.914914:0:13808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:53607:x1871938520018816:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 1297us (3741us total) trans 0 rc 301/301 00000100:00100000:2.0:1785220585.914922:0:13808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91796 00010000:00010000:3.0:1785220585.914943:0:13921:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69000/0xf62499ab927a5e87 lrc: 3/1,0 mode: --/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13921 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.914944:0:13808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91798 00000100:00100000:2.0:1785220585.914955:0:13808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:5644b1e7-55e7-4421-a7e4-714e45fd656f+689:53889:x1871938520020352:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:3.0:1785220585.914957:0:13921:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69000/0xf62499ab927a5e87 lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13921 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.914965:0:13808:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220585.914969:0:13921:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69000/0xf62499ab927a5e87 lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13921 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.914977:0:13808:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93200/0xf62499ab927a5e8e lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd65 expref: -99 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1785220585.915013:0:13921:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000404:0x7c5:0x0]: returning size 4096 00000004:00200000:3.0:1785220585.915017:0:13921:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:2.0:1785220585.915034:0:13808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93c00/0xf62499ab927a5e95 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13808 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.915045:0:13808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93c00/0xf62499ab927a5e95 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13808 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.915049:0:13921:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68a00/0xf62499ab927a5e80 lrc: 0/0,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd5e expref: 691 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.915059:0:13808:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93c00/0xf62499ab927a5e95 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13808 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.915068:0:13921:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69000/0xf62499ab927a5e87 lrc: 3/0,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd5e expref: 690 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.915076:0:13921:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x7c5:0x0].0x0 (ffff960d8e268b00) 00010000:00010000:3.0:1785220585.915079:0:13921:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1785220585.915083:0:13921:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea69000, rc 0) 00000100:00100000:3.0:1785220585.915134:0:13921:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:5644b1e7-55e7-4421-a7e4-714e45fd656f+690:55377:x1871938520020224:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 287us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.915140:0:13921:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91797 00000004:00200000:2.0:1785220585.915616:0:13808:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb05:0x0]: returning size 4096 00000004:00200000:2.0:1785220585.915621:0:13808:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:2.0:1785220585.915652:0:13808:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93200/0xf62499ab927a5e8e lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd65 expref: 690 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.915667:0:13808:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93c00/0xf62499ab927a5e95 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd65 expref: 689 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.915673:0:13808:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d94c6c900) 00010000:00010000:2.0:1785220585.915675:0:13808:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1785220585.915678:0:13808:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b93c00, rc 0) 00000100:00100000:2.0:1785220585.915776:0:13808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:5644b1e7-55e7-4421-a7e4-714e45fd656f+689:53889:x1871938520020352:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 821us (1096us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220585.915784:0:13808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91798 00000100:00100000:3.0:1785220585.921431:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.921534:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220585.921729:0:13807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520020736 00000100:00100000:1.0:1785220585.921729:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520020608 00000100:00100000:2.0:1785220585.921742:0:13807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520020736 00000100:00100000:1.0:1785220585.921742:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520020608 00000100:00100000:2.0:1785220585.921815:0:13807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91799 00000100:00100000:1.0:1785220585.921816:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 25171 00000100:00100000:2.0:1785220585.921830:0:13807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:5644b1e7-55e7-4421-a7e4-714e45fd656f+473:53673:x1871938520020736:12345-192.168.201.51@tcp:101:lfs.0 00000100:00100000:1.0:1785220585.921830:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+473:53627:x1871938520020608:12345-192.168.201.51@tcp:35:touch.0 00010000:00010000:2.0:1785220585.921900:0:13807:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1785220585.921928:0:13807:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92e00/0xf62499ab927a5e9c lrc: 2/0,0 mode: --/CR res: [0x240000400:0x49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd6c expref: -99 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.922014:0:13807:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5ea3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13807 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.922029:0:13807:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5ea3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.922041:0:13807:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5ea3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13807 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1785220585.922073:0:13807:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x49:0x0]: returning size 4096 00000004:00200000:2.0:1785220585.922079:0:13807:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000100:00100000:1.0:1785220585.922222:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+475:53627:x1871938520020608:12345-192.168.201.51@tcp:35:touch.0 Request processed in 395us (798us total) trans 4294992693 rc 0/0 00000100:00100000:1.0:1785220585.922230:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 25171 00000100:00100000:1.0:1785220585.924409:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220585.924474:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220585.924636:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.924900:0:13847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520021120 00000100:00100000:3.0:1785220585.924911:0:13847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520021120 00000100:00100000:3.0:1785220585.924958:0:13847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520021376 00000100:00100000:3.0:1785220585.924962:0:13847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520021376 00000100:00100000:3.0:1785220585.924989:0:13847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91800 00000100:00100000:3.0:1785220585.925000:0:13847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_018:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:53607:x1871938520021120:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:3.0:1785220585.925018:0:13847:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220585.925040:0:13847:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68600/0xf62499ab927a5eaa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd7a expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925157:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925172:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925183:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925196:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925203:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925209:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925250:0:13847:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925257:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925264:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925271:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925277:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.925281:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925289:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925300:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:3.0:1785220585.925303:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.925307:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925314:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a5eb1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925327:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925333:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925339:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925344:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.925347:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925357:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925365:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0xdcc87686 (ffff960d8e268500) 00010000:00010000:3.0:1785220585.925367:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.925370:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925376:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a5eb8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925412:0:13847:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68600/0xf62499ab927a5eaa lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd7a expref: 589 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925422:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:3.0:1785220585.925424:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1785220585.925427:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68600/0xf62499ab927a5eaa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd7a expref: 589 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925436:0:13847:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea68600, rc 0) 00000100:00100000:3.0:1785220585.925813:0:13847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_018:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:53607:x1871938520021120:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 759us (1358us total) trans 0 rc 301/301 00000100:00100000:3.0:1785220585.925821:0:13847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91800 00000100:00100000:3.0:1785220585.925842:0:13847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91801 00000100:00100000:3.0:1785220585.925850:0:13847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8add7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_018:5644b1e7-55e7-4421-a7e4-714e45fd656f+689:55377:x1871938520021376:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:3.0:1785220585.925858:0:13847:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220585.925867:0:13847:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69800/0xf62499ab927a5ebf lrc: 2/0,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd81 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925924:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 3/1,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925933:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 3/1,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925941:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 3/1,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925954:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 3/1,0 mode: --/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925961:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925967:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925991:0:13847:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 4/1,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.925997:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 4/1,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926002:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 3/0,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926008:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 4/0,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926013:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68e00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.926017:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 4/0,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926024:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 5/0,0 mode: CR/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926032:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x7c5:0x0].0x0 (ffff960d8e268b00) 00010000:00010000:3.0:1785220585.926035:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.926038:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 3/0,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926045:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68e00/0xf62499ab927a5ec6 lrc: 0/0,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926055:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 3/1,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926060:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 2/0,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926066:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 3/0,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926071:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.926073:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 3/0,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926080:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 4/0,0 mode: PR/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926108:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x7c5:0x0].0x59d92b11 (ffff960d8e268300) 00010000:00010000:3.0:1785220585.926111:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.926114:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 2/0,0 mode: --/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926120:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68c00/0xf62499ab927a5ecd lrc: 0/0,0 mode: --/PR res: [0x200000404:0x7c5:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926153:0:13847:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69800/0xf62499ab927a5ebf lrc: 1/0,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd81 expref: 690 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926162:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x7c5:0x0].0x0 (ffff960d8e268b00) 00010000:00010000:3.0:1785220585.926164:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.926167:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69800/0xf62499ab927a5ebf lrc: 0/0,0 mode: --/CR res: [0x200000404:0x7c5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd81 expref: 690 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.926175:0:13847:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea69800, rc 0) 00000100:00100000:3.0:1785220585.926199:0:13847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8add7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_018:5644b1e7-55e7-4421-a7e4-714e45fd656f+689:55377:x1871938520021376:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 350us (1727us total) trans 0 rc 301/301 00000100:00100000:3.0:1785220585.926206:0:13847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91801 00000100:00100000:1.0:1785220585.926872:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220585.926906:0:13805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520021504 00000100:00100000:1.0:1785220585.926912:0:13805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520021504 00000100:00100000:1.0:1785220585.926942:0:13805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520020992 00000100:00100000:1.0:1785220585.926945:0:13805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520020992 00000100:00100000:1.0:1785220585.926960:0:13805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91802 00000100:00100000:1.0:1785220585.926966:0:13805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c299f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:53602:x1871938520021504:12345-192.168.201.51@tcp:36:rm.0 00010000:00010000:1.0:1785220585.927008:0:13805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1785220585.927025:0:13805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d93cce400/0xf62499ab92726742 lrc: 0/0,0 mode: --/CR res: [0x200000404:0x768:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5c626 expref: 588 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.927043:0:13805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94a0f400/0xf62499ab92724dcd lrc: 0/0,0 mode: --/PR res: [0x200000404:0x768:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5bacd expref: 587 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.927060:0:13805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x768:0x0].0x0 (ffff960d934ec400) 00010000:00010000:1.0:1785220585.927068:0:13805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220585.927234:0:13805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.927243:0:13805:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.927253:0:13805:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.927273:0:13805:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814464:0@lo:101:mdt00_006.0 00000100:00100000:1.0:1785220585.927319:0:13805:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.927397:0:13805:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.927415:0:13810:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91803 00000100:00100000:1.0:1785220585.927422:0:13810:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c299880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:5644b1e7-55e7-4421-a7e4-714e45fd656f+689:53670:x1871938520020992:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:1.0:1785220585.927431:0:13810:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.927442:0:13810:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a5edb lrc: 2/0,0 mode: --/CR res: [0x200000404:0x806:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd73 expref: -99 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1785220585.927561:0:13810:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000404:0x806:0x0]: returning size 4096 00010000:00010000:1.0:1785220585.927645:0:13810:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a5edb lrc: 1/0,0 mode: --/CR res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd73 expref: 691 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.927655:0:13810:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x806:0x0].0x0 (ffff960d8a305700) 00010000:00010000:1.0:1785220585.927658:0:13810:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1785220585.927661:0:13810:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a5edb lrc: 0/0,0 mode: --/CR res: [0x200000404:0x806:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd73 expref: 691 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.927674:0:13810:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496200, rc 0) 00000100:00100000:1.0:1785220585.927716:0:13810:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c299880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:5644b1e7-55e7-4421-a7e4-714e45fd656f+690:53670:x1871938520020992:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 295us (847us total) trans 4295000089 rc 301/301 00000100:00100000:1.0:1785220585.927723:0:13810:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91803 00000100:00100000:3.0:1785220585.927970:0:13845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814464 00000100:00100000:3.0:1785220585.927975:0:13845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814464 00000100:00100000:3.0:1785220585.928052:0:13845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12568 00000100:00100000:3.0:1785220585.928060:0:13845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c29a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:13805:x1871938510814464:12345-0@lo:101:mdt00_006.0 00010000:00010000:3.0:1785220585.928069:0:13845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220585.928082:0:13845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69200/0xf62499ab927a5ee9 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a5ed4 expref: -99 pid: 13845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.928134:0:13845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69200/0xf62499ab927a5ee9 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a5ed4 expref: 6 pid: 13845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.928150:0:13845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69200/0xf62499ab927a5ee9 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a5ed4 expref: 6 pid: 13845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.928157:0:13845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.928159:0:13845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.928161:0:13845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea69200, rc 0) 00000100:00100000:3.0:1785220585.928197:0:13845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c29a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:13805:x1871938510814464:12345-0@lo:101:mdt00_006.0 Request processed in 138us (876us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.928203:0:13845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12568 00000100:00100000:1.0:1785220585.928274:0:13805:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814464:0@lo:101:mdt00_006.0 00010000:00010000:1.0:1785220585.928290:0:13805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.928298:0:13807:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92e00/0xf62499ab927a5e9c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd6c expref: 475 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.928304:0:13805:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.928325:0:13807:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92200/0xf62499ab927a5ea3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd6c expref: 474 pid: 13807 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220585.928332:0:13805:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00010000:00010000:2.0:1785220585.928337:0:13807:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x49:0x0].0x0 (ffff960d9843bb00) 00010000:00010000:2.0:1785220585.928341:0:13807:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1785220585.928344:0:13807:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b92200, rc 0) 00010000:00010000:1.0:1785220585.928357:0:13805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497200/0xf62499ab927a5ef0 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.928365:0:13805:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497200/0xf62499ab927a5ef0 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.928374:0:13807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:5644b1e7-55e7-4421-a7e4-714e45fd656f+474:53673:x1871938520020736:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 6549us (6838us total) trans 0 rc 0/0 00010000:00010000:1.0:1785220585.928375:0:13805:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497200/0xf62499ab927a5ef0 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1785220585.928389:0:13807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91799 00000100:00100000:1.0:1785220585.928399:0:13805:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814592:0@lo:101:mdt00_006.0 00000100:00100000:1.0:1785220585.928427:0:13805:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.928450:0:13805:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.928682:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814592 00000100:00100000:1.0:1785220585.928695:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814592 00000100:00100000:1.0:1785220585.928771:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12569 00000100:00100000:1.0:1785220585.928778:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c299500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:13805:x1871938510814592:12345-0@lo:101:mdt00_006.0 00010000:00010000:1.0:1785220585.928787:0:6531:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.928799:0:6531:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5ef7 lrc: 2/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a5ef0 expref: -99 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.928814:0:6531:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5ef7 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a5ef0 expref: 7 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.928824:0:6531:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497800/0xf62499ab927a5ef7 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a5ef0 expref: 7 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.928831:0:6531:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x7a94fdef (ffff960d8a305b00) 00010000:00010000:1.0:1785220585.928834:0:6531:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.928836:0:6531:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98497800, rc 0) 00000100:00100000:1.0:1785220585.928862:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c299500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:13805:x1871938510814592:12345-0@lo:101:mdt00_006.0 Request processed in 85us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.928867:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12569 00000100:00100000:1.0:1785220585.928888:0:13805:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814592:0@lo:101:mdt00_006.0 00010000:00010000:1.0:1785220585.928897:0:13805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497200/0xf62499ab927a5ef0 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5ef7 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.928906:0:13805:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497200/0xf62499ab927a5ef0 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5ef7 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1785220585.928918:0:13805:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00000100:00100000:1.0:1785220585.928945:0:13805:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814720:0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1785220585.928967:0:13805:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.928981:0:13805:0:(client.c:2702:ptlrpc_set_wait()) set ffff960eba869480 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.928996:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814720 00000100:00100000:1.0:1785220585.928999:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814720 00000100:00100000:1.0:1785220585.929022:0:13815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12570 00000100:00100000:1.0:1785220585.929028:0:13815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c298700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:13805:x1871938510814720:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1785220585.929079:0:13815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c298700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:13805:x1871938510814720:12345-0@lo:1000:mdt00_006.0 Request processed in 52us (111us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.929105:0:13815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12570 00000100:00100000:1.0:1785220585.929130:0:13805:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814720:0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1785220585.929163:0:13805:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814848:0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1785220585.929186:0:13805:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.929200:0:13805:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d896840 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.929211:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814848 00000100:00100000:1.0:1785220585.929214:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814848 00000100:00100000:1.0:1785220585.929236:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12571 00000100:00100000:1.0:1785220585.929242:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c29b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:13805:x1871938510814848:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1785220585.929299:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c29b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:13805:x1871938510814848:12345-0@lo:1000:mdt00_006.0 Request processed in 57us (111us total) trans 0 rc 1/1 00000100:00100000:1.0:1785220585.929304:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12571 00000100:00100000:1.0:1785220585.929327:0:13805:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814848:0@lo:1000:mdt00_006.0 00010000:00010000:1.0:1785220585.929361:0:13805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 3/0,1 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929369:0:13805:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 3/0,1 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929377:0:13805:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 3/0,1 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.929388:0:13805:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814976:0@lo:101:mdt00_006.0 00000100:00100000:1.0:1785220585.929406:0:13805:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.929417:0:13805:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e811cac00 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.929426:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510814976 00000100:00100000:1.0:1785220585.929429:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510814976 00000100:00100000:1.0:1785220585.929444:0:13815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12572 00000100:00100000:1.0:1785220585.929449:0:13815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c299c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:13805:x1871938510814976:12345-0@lo:101:mdt00_006.0 00010000:00010000:1.0:1785220585.929466:0:13815:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.929477:0:13815:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5f05 lrc: 2/0,0 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a5efe expref: -99 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929493:0:13815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5f05 lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a5efe expref: 8 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929501:0:13815:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5f05 lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a5efe expref: 8 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929508:0:13815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x768:0x0].0x0 (ffff960d8a305a00) 00010000:00010000:1.0:1785220585.929511:0:13815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1785220585.929513:0:13815:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496a00, rc 0) 00000100:00100000:1.0:1785220585.929538:0:13815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c299c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:13805:x1871938510814976:12345-0@lo:101:mdt00_006.0 Request processed in 89us (131us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.929544:0:13815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12572 00000100:00100000:1.0:1785220585.929569:0:13805:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:13805:1871938510814976:0@lo:101:mdt00_006.0 00010000:00010000:1.0:1785220585.929577:0:13805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 4/0,1 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929591:0:13805:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 4/0,1 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929611:0:13805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 3/0,1 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929620:0:13805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 3/0,1 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.929627:0:13805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 3/0,1 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1785220585.929899:0:13805:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x2:0x0].1645, 656 off1034944 00000040:00080000:1.0:1785220585.929938:0:13805:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x2:0x0].1650, 736 off1289792 00000004:00080000:1.0:1785220585.929943:0:13805:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff960e811cac00 with version 3228 00000004:00080000:1.0:1785220585.929960:0:7832:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff960e85e026c0 version 3228 rpc_version 3228 00000020:00100000:1.0:1785220585.929980:0:7832:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1785220585.929987:0:7832:0:(client.c:807:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff960d8c298000 x1871938510815232/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1968/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1785220585.930000:0:7832:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7832:1871938510815232:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1785220585.930026:0:7832:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.930050:0:7832:0:(client.c:2702:ptlrpc_set_wait()) set ffff960e9aedcd80 going to sleep for 11 seconds 00000100:00100000:3.0:1785220585.930243:0:13845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510815232 00000100:00100000:3.0:1785220585.930247:0:13845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510815232 00000100:00100000:3.0:1785220585.930270:0:13845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12573 00000100:00100000:3.0:1785220585.930275:0:13845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c298380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:7832:x1871938510815232:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1785220585.930553:0:13845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c298380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:7832:x1871938510815232:12345-0@lo:1000:osp_up1-0.0 Request processed in 276us (525us total) trans 4294992694 rc 0/0 00000100:00100000:3.0:1785220585.930560:0:13845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12573 00000100:00100000:0.0:1785220585.930766:0:7832:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c298000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7832:1871938510815232:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1785220585.930833:0:13805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000404:0x768:0x0]: returning size 4096 00010000:00010000:3.0:1785220585.930851:0:13805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 3/0,1 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930860:0:13805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 2/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930868:0:13805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930874:0:13805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.930878:0:13805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930888:0:13805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 4/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930902:0:13805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x768:0x0].0x0 (ffff960d8a305600) 00010000:00010000:3.0:1785220585.930905:0:13805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.930910:0:13805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 2/0,0 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930917:0:13805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a5f0c lrc: 0/0,0 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930935:0:13805:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 4/0,1 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930944:0:13805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 4/0,1 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930951:0:13805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930965:0:13805:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1785220585.930976:0:13805:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00010000:00010000:3.0:1785220585.930981:0:13805:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930988:0:13805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.930995:0:13805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.931015:0:13805:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1785220585.931019:0:13805:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x1:0x0] 00010000:00010000:3.0:1785220585.931024:0:13805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497200/0xf62499ab927a5ef0 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5ef7 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.931032:0:13805:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98497200/0xf62499ab927a5ef0 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0x7a94fdef bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a5ef7 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.931050:0:10345:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1785220585.931058:0:10345:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 2/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.931066:0:10345:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496600 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:1.0:1785220585.931071:0:10345:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1785220585.931080:0:10345:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 3/0,0 mode: EX/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.931219:0:10345:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 2/0,0 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.931228:0:10345:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1785220585.931246:0:10345:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 1/0,0 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.931248:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1785220585.931257:0:10345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496600/0xf62499ab927a5efe lrc: 0/0,0 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5f05 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.931265:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.931272:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1785220585.931276:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.931277:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496800 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:1.0:1785220585.931278:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.931285:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1785220585.931294:0:6515:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.931300:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8c29b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510815360:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1785220585.931318:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.931325:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1785220585.931334:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 1/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.931336:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1785220585.931345:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff960d98496800/0xf62499ab927a5ed4 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a5ee9 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.931364:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.931367:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1785220585.931370:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220585.931377:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1785220585.931385:0:3654:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510815488:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.931396:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510815360 00000100:00100000:0.0:1785220585.931402:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510815360 00000100:00100000:1.0:1785220585.931411:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220585.931440:0:14229:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510815488 00000100:00100000:0.0:1785220585.931444:0:14229:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510815488 00000100:00100000:1.0:1785220585.931448:0:13853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14254 00000100:00100000:1.0:1785220585.931459:0:13853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c29a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:lustre-MDT0000-mdtlov_UUID+10:3654:x1871938510815360:12345-0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1785220585.931471:0:14229:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14255 00010000:00010000:1.0:1785220585.931472:0:13853:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1785220585.931482:0:14229:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97b39500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:lustre-MDT0000-mdtlov_UUID+10:3654:x1871938510815488:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1785220585.931490:0:13853:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5f05 lrc: 0/0,0 mode: --/EX res: [0x200000404:0x768:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab927a5efe expref: 10 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.931493:0:14229:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1785220585.931509:0:14229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69200/0xf62499ab927a5ee9 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab927a5ed4 expref: 9 pid: 13845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.931510:0:13853:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x768:0x0].0x0 (ffff960d8a305a00) 00010000:00010000:1.0:1785220585.931521:0:13853:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1785220585.931528:0:14229:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:0.0:1785220585.931534:0:14229:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1785220585.931554:0:13853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c29a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_007:lustre-MDT0000-mdtlov_UUID+8:3654:x1871938510815360:12345-0@lo:103:ldlm_bl.0 Request processed in 98us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220585.931563:0:13853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14254 00000100:00100000:0.0:1785220585.931571:0:14229:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97b39500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_011:lustre-MDT0000-mdtlov_UUID+8:3654:x1871938510815488:12345-0@lo:103:ldlm_bl.0 Request processed in 90us (159us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.931578:0:14229:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14255 00000100:00100000:1.0:1785220585.931596:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510815488:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1785220585.931607:0:3654:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8c29b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1871938510815360:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1785220585.931673:0:13805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c299f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:53602:x1871938520021504:12345-192.168.201.51@tcp:36:rm.0 Request processed in 4154us (6487us total) trans 4295000090 rc 0/0 00000100:00100000:3.0:1785220585.931683:0:13805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91802 00000100:00100000:1.0:1785220585.935977:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220585.936025:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220585.936045:0:13919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520021888 00000100:00100000:0.0:1785220585.936058:0:13919:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520021888 00000100:00100000:1.0:1785220585.936204:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220585.937046:0:13824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520022656 00000100:00100000:2.0:1785220585.937056:0:13824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520022656 00000100:00100000:2.0:1785220585.937193:0:13824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 25172 00000100:00100000:2.0:1785220585.937204:0:13824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_005:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:53670:x1871938520022656:12345-192.168.201.51@tcp:35:lfs.0 00000100:00100000:2.0:1785220585.937654:0:13824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_005:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53670:x1871938520022656:12345-192.168.201.51@tcp:35:lfs.0 Request processed in 452us (1628us total) trans 4295000091 rc 0/0 00000100:00100000:2.0:1785220585.937660:0:13824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 25172 00000100:00100000:2.0:1785220585.937893:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520022784 00000100:00100000:2.0:1785220585.937899:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520022784 00000100:00100000:0.0:1785220585.938022:0:13919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91804 00000100:00100000:0.0:1785220585.938033:0:13919:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8c29b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_024:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53889:x1871938520021888:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:0.0:1785220585.938046:0:13919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1785220585.938073:0:13846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91805 00010000:00010000:0.0:1785220585.938073:0:13919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3600/0xf62499ab927a5f13 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd88 expref: -99 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.938106:0:13846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_017:5644b1e7-55e7-4421-a7e4-714e45fd656f+693:53673:x1871938520022784:12345-192.168.201.51@tcp:101:lfs.0 00010000:00010000:1.0:1785220585.938115:0:13846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.938130:0:13846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a5f1a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd8f expref: -99 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938185:0:13919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 3/1,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938212:0:13919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938225:0:13919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938239:0:13919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938246:0:13919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.938252:0:13846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5f2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938254:0:13919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.938260:0:13846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5f2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.938276:0:13846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5f2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1785220585.938315:0:13846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 4096 00010000:00010000:0.0:1785220585.938318:0:13919:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1785220585.938320:0:13846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1785220585.938330:0:13919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938337:0:13919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938344:0:13919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.938346:0:13846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a5f1a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd8f expref: 695 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938367:0:13919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3e00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.938372:0:13919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.938373:0:13846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496600/0xf62499ab927a5f2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd8f expref: 694 pid: 13846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938390:0:13919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.938392:0:13846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960d8a305a00) 00010000:00010000:1.0:1785220585.938398:0:13846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1785220585.938402:0:13846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496600, rc 0) 00010000:00010000:0.0:1785220585.938407:0:13919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d94c6c900) 00010000:00010000:0.0:1785220585.938411:0:13919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.938415:0:13919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938420:0:13919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3e00/0xf62499ab927a5f21 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938432:0:13919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938439:0:13919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.938444:0:13846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_017:5644b1e7-55e7-4421-a7e4-714e45fd656f+694:53673:x1871938520022784:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 359us (2237us total) trans 0 rc 0/0 00010000:00010000:0.0:1785220585.938452:0:13919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938459:0:13919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2000 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.938461:0:13919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220585.938466:0:13846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91805 00010000:00010000:0.0:1785220585.938470:0:13919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938482:0:13919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0xdcc87686 (ffff960d959b4c00) 00010000:00010000:0.0:1785220585.938486:0:13919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.938489:0:13919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938496:0:13919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2000/0xf62499ab927a5f28 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb05:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938531:0:13919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3600/0xf62499ab927a5f13 lrc: 1/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd88 expref: 693 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938540:0:13919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb05:0x0].0x0 (ffff960d94c6c900) 00010000:00010000:0.0:1785220585.938543:0:13919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.938547:0:13919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3600/0xf62499ab927a5f13 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb05:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd88 expref: 693 pid: 13919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.938556:0:13919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3600, rc 0) 00000100:00100000:0.0:1785220585.938581:0:13919:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8c29b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_024:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53889:x1871938520021888:12345-192.168.201.51@tcp:101:lfs.0 Request processed in 551us (2615us total) trans 0 rc 301/301 00000100:00100000:0.0:1785220585.938588:0:13919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91804 00000100:00100000:2.0:1785220585.942272:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220585.942353:0:13872:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520023424 00000100:00100000:1.0:1785220585.942373:0:13872:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520023424 00000100:00100000:1.0:1785220585.942439:0:13872:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91806 00000100:00100000:1.0:1785220585.942450:0:13872:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:5644b1e7-55e7-4421-a7e4-714e45fd656f+474:53631:x1871938520023424:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:1.0:1785220585.942473:0:13872:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220585.942492:0:13872:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f36 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fd96 expref: -99 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942585:0:13872:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13872 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942601:0:13872:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942611:0:13872:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942632:0:13872:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13872 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942638:0:13872:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942645:0:13872:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942697:0:13872:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942705:0:13872:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942711:0:13872:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942717:0:13872:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942723:0:13872:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496e00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.942727:0:13872:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942734:0:13872:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942745:0:13872:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220585.942748:0:13872:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.942753:0:13872:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942760:0:13872:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5f3d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942770:0:13872:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942775:0:13872:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942781:0:13872:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942785:0:13872:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.942788:0:13872:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942793:0:13872:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942799:0:13872:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xf7a5b27a (ffff960d8a305e00) 00010000:00010000:1.0:1785220585.942801:0:13872:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.942804:0:13872:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942809:0:13872:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5f44 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xf7a5b27a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942846:0:13872:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f36 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd96 expref: 475 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942861:0:13872:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220585.942863:0:13872:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.942867:0:13872:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f36 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fd96 expref: 475 pid: 13872 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.942875:0:13872:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496200, rc 0) 00000100:00100000:1.0:1785220585.944190:0:13872:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:5644b1e7-55e7-4421-a7e4-714e45fd656f+474:53631:x1871938520023424:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 1734us (1920us total) trans 0 rc 301/301 00000100:00100000:1.0:1785220585.944202:0:13872:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91806 00000100:00100000:2.0:1785220585.954868:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220585.955167:0:8466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520024576 00000100:00100000:0.0:1785220585.955178:0:8466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520024576 00000100:00100000:0.0:1785220585.955245:0:8466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91807 00000100:00100000:0.0:1785220585.955255:0:8466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:53752:x1871938520024576:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:0.0:1785220585.955282:0:8466:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1785220585.955480:0:8466:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3c00/0xf62499ab927a5f4b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdab expref: -99 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955581:0:8466:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955597:0:8466:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955610:0:8466:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955624:0:8466:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955631:0:8466:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955638:0:8466:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955704:0:8466:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a5f60 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xac1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955713:0:8466:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a5f60 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xac1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955720:0:8466:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a5f60 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xac1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1785220585.955742:0:8466:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a5f60 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xac1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955757:0:8466:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955765:0:8466:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955770:0:8466:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955776:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955781:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.955786:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955792:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955804:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:0.0:1785220585.955807:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.955811:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955818:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a5f52 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955832:0:8466:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955837:0:8466:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955843:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955848:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2a00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.955851:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955856:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955871:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x59d92b11 (ffff960d959b4300) 00010000:00010000:0.0:1785220585.955873:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.955876:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955882:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a5f59 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8466 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955913:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3c00/0xf62499ab927a5f4b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdab expref: 589 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955926:0:8466:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a5f60 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xac1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdab expref: 588 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.955934:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xac1:0x0].0x0 (ffff960d8d401c00) 00010000:00010000:0.0:1785220585.955936:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1785220585.955938:0:8466:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be2600, rc 0) 00000100:00100000:0.0:1785220585.956006:0:8466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:53752:x1871938520024576:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 752us (1146us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.956014:0:8466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91807 00000100:00100000:2.0:1785220585.957558:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.957748:0:13826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520025472 00000100:00100000:3.0:1785220585.957756:0:13826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520025472 00000100:00100000:3.0:1785220585.957806:0:13826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91808 00000100:00100000:3.0:1785220585.957817:0:13826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_016:5644b1e7-55e7-4421-a7e4-714e45fd656f+474:53633:x1871938520025472:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:3.0:1785220585.957836:0:13826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220585.957855:0:13826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a5f67 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdb2 expref: -99 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.957940:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.957952:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.957964:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.957980:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.957987:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.957993:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.958062:0:13826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68e00/0xf62499ab927a5f7c lrc: 3/1,0 mode: --/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13826 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.958070:0:13826:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68e00/0xf62499ab927a5f7c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.958077:0:13826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68e00/0xf62499ab927a5f7c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1785220585.959502:0:13826:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x98e:0x0]: returning size 4096 00000004:00200000:3.0:1785220585.959509:0:13826:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1785220585.959535:0:13826:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68e00/0xf62499ab927a5f7c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959551:0:13826:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959558:0:13826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959565:0:13826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959576:0:13826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959581:0:13826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.959585:0:13826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959592:0:13826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959604:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220585.959607:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.959612:0:13826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959618:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5f6e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959630:0:13826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959637:0:13826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959644:0:13826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959650:0:13826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220585.959653:0:13826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959664:0:13826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959672:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xd964d85e (ffff960d8e268300) 00010000:00010000:3.0:1785220585.959674:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220585.959677:0:13826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959684:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5f75 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13826 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959735:0:13826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a5f67 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdb2 expref: 476 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959750:0:13826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68e00/0xf62499ab927a5f7c lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdb2 expref: 475 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220585.959758:0:13826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x98e:0x0].0x0 (ffff960d8e268500) 00010000:00010000:3.0:1785220585.959761:0:13826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220585.959764:0:13826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea68e00, rc 0) 00000100:00100000:3.0:1785220585.959833:0:13826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_016:5644b1e7-55e7-4421-a7e4-714e45fd656f+475:53633:x1871938520025472:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 2019us (2282us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220585.959841:0:13826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91808 00000100:00100000:2.0:1785220585.967557:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220585.972518:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520025600 00000100:00100000:3.0:1785220585.972533:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520025600 00000100:00100000:0.0:1785220585.972907:0:13921:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91809 00000100:00100000:0.0:1785220585.972918:0:13921:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:5644b1e7-55e7-4421-a7e4-714e45fd656f+475:53633:x1871938520025600:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:0.0:1785220585.972940:0:13921:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1785220585.972961:0:13921:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3c00/0xf62499ab927a5f83 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdb9 expref: -99 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973056:0:13921:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13921 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973070:0:13921:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973082:0:13921:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973119:0:13921:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13921 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973139:0:13921:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973147:0:13921:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973234:0:13921:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5f98 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x96b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13921 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973242:0:13921:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5f98 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x96b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973248:0:13921:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5f98 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x96b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1785220585.973531:0:13921:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x96b:0x0]: returning size 4096 00000004:00200000:0.0:1785220585.973536:0:13921:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1785220585.973557:0:13921:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5f98 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x96b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973571:0:13921:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973578:0:13921:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973585:0:13921:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973591:0:13921:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973598:0:13921:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2a00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.973601:0:13921:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973608:0:13921:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973619:0:13921:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:0.0:1785220585.973623:0:13921:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.973628:0:13921:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973634:0:13921:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2a00/0xf62499ab927a5f8a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973652:0:13921:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973658:0:13921:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973664:0:13921:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973668:0:13921:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220585.973671:0:13921:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973676:0:13921:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973682:0:13921:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xa5ec40c9 (ffff960d959b4300) 00010000:00010000:0.0:1785220585.973694:0:13921:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220585.973698:0:13921:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973704:0:13921:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be2800/0xf62499ab927a5f91 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13921 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973733:0:13921:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3c00/0xf62499ab927a5f83 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdb9 expref: 477 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973746:0:13921:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5f98 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x96b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdb9 expref: 476 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220585.973759:0:13921:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x96b:0x0].0x0 (ffff960d959b4c00) 00010000:00010000:0.0:1785220585.973762:0:13921:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1785220585.973774:0:13921:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3600, rc 0) 00000100:00100000:0.0:1785220585.973848:0:13921:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53633:x1871938520025600:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 932us (6301us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220585.973855:0:13921:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91809 00000100:00100000:1.0:1785220585.976845:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220585.984952:0:14239:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520026880 00000100:00100000:0.0:1785220585.984966:0:14239:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520026880 00000100:00100000:2.0:1785220585.985178:0:14239:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 14256 00000100:00100000:2.0:1785220585.985191:0:14239:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93608a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:2405:x1871938520026880:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1785220585.985404:0:14239:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1785220585.985430:0:14239:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a5f60 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xac1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdab expref: 588 pid: 8466 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220585.985453:0:14239:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xac1:0x0].0x0 (ffff960d8d401c00) 00010000:00010000:2.0:1785220585.985458:0:14239:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1785220585.985530:0:14239:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_014:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2405:x1871938520026880:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 342us (8694us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220585.985538:0:14239:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 14256 00000100:00100000:2.0:1785220585.995650:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220585.995763:0:6540:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520027136 00000100:00100000:1.0:1785220585.995774:0:6540:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520027136 00000100:00100000:1.0:1785220585.995904:0:6540:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 2912 00000100:00100000:1.0:1785220585.995916:0:6540:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53633:x1871938520027136:12345-192.168.201.51@tcp:36:mrename.0 00010000:00010000:1.0:1785220585.995972:0:6540:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1785220585.996000:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d94be3600/0xf62499ab927a5f98 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x96b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdb9 expref: 476 pid: 13921 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996021:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x96b:0x0].0x0 (ffff960d959b4c00) 00010000:00010000:1.0:1785220585.996038:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68e00/0xf62499ab927a5f7c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdb2 expref: 475 pid: 13826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996048:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x98e:0x0].0x0 (ffff960d8e268500) 00010000:00010000:1.0:1785220585.996056:0:6540:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220585.996082:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996145:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996155:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996171:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5fa6 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996178:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5fa6 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996184:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5fa6 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996244:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5fad lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0xd964d85e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996253:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5fad lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996260:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5fad lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996363:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996372:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996378:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996394:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496000/0xf62499ab927a5fbb lrc: 3/0,1 mode: --/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996400:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496000/0xf62499ab927a5fbb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.996406:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496000/0xf62499ab927a5fbb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1785220585.996827:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].1244, 984 off893472 00000040:00080000:1.0:1785220585.996891:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].1248, 1184 off1073024 00000004:00080000:1.0:1785220585.996896:0:6540:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff960d8df013c0 with version 2433 00000004:00080000:1.0:1785220585.996923:0:7470:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff960ea686e240 version 2433 rpc_version 2433 00000020:00100000:1.0:1785220585.996954:0:7470:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1785220585.996964:0:7470:0:(client.c:807:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff960d9360bb80 x1871938510815744/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2160/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1785220585.996985:0:7470:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360bb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510815744:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1785220585.997045:0:7470:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220585.997122:0:7470:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01b40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220585.997141:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510815744 00000100:00100000:1.0:1785220585.997145:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510815744 00000100:00100000:1.0:1785220585.997183:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12574 00000100:00100000:1.0:1785220585.997190:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93608000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7470:x1871938510815744:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1785220585.997339:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93608000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7470:x1871938510815744:12345-0@lo:1000:osp_up0-1.0 Request processed in 149us (293us total) trans 4295000092 rc 0/0 00000100:00100000:1.0:1785220585.997347:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12574 00000100:00100000:1.0:1785220585.997398:0:7470:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360bb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7470:1871938510815744:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1785220585.997418:0:6540:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x96b:0x0]: returning size 4096 00010000:00010000:1.0:1785220585.997437:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997445:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997453:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997459:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496a00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.997463:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997470:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997482:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x96b:0x0].0x0 (ffff960d8a305d00) 00010000:00010000:1.0:1785220585.997484:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220585.997489:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997494:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496a00/0xf62499ab927a5fb4 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x96b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1785220585.997510:0:6540:0:(mdt_handler.c:4478:mdt_save_lock()) ### save lock request 000000005c305c56 reply state 00000000b12eb894 transno 4294992695 ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496000/0xf62499ab927a5fbb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997521:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496000/0xf62499ab927a5fbb lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x98e:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997527:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x98e:0x0].0x0 (ffff960d8a305100) 00010000:00010000:1.0:1785220585.997529:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00000100:00100000:1.0:1785220585.997532:0:6540:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1785220585.997541:0:6540:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997547:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997552:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997557:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997562:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496200 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220585.997565:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997571:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997578:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220585.997580:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220585.997583:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 3/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997588:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496200/0xf62499ab927a5f9f lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1785220585.997597:0:6540:0:(mdt_handler.c:4478:mdt_save_lock()) ### save lock request 000000005c305c56 reply state 00000000b12eb894 transno 4294992695 ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5fa6 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997604:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497400/0xf62499ab927a5fa6 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997609:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xa5ec40c9 (ffff960d8a305e00) 00010000:00010000:1.0:1785220585.997611:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1785220585.997613:0:6540:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00010000:1.0:1785220585.997616:0:6540:0:(mdt_handler.c:4478:mdt_save_lock()) ### save lock request 000000005c305c56 reply state 00000000b12eb894 transno 4294992695 ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5fad lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997623:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496e00/0xf62499ab927a5fad lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x1:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220585.997629:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xd964d85e (ffff960d8a305700) 00010000:00010000:1.0:1785220585.997631:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1785220585.997632:0:6540:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:1.0:1785220585.997698:0:6540:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53633:x1871938520027136:12345-192.168.201.51@tcp:36:mrename.0 Request processed in 1772us (2045us total) trans 4294992695 rc 0/0 00000100:00100000:1.0:1785220585.997706:0:6540:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 2912 00000100:00080000:3.0:1785220585.998422:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1785220586.001614:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220586.001732:0:13850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520027264 00000100:00100000:3.0:1785220586.001742:0:13850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520027264 00000100:00100000:3.0:1785220586.001806:0:13850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91810 00000100:00100000:3.0:1785220586.001817:0:13850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_019:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53552:x1871938520027264:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:3.0:1785220586.001848:0:13850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220586.001870:0:13850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a5fc2 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdc7 expref: -99 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.001979:0:13850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.001998:0:13850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002013:0:13850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002030:0:13850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002038:0:13850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002045:0:13850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002121:0:13850:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002128:0:13850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002134:0:13850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002145:0:13850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002152:0:13850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220586.002158:0:13850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002167:0:13850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002179:0:13850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220586.002182:0:13850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220586.002187:0:13850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002200:0:13850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a5fc9 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002220:0:13850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002226:0:13850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002231:0:13850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002236:0:13850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220586.002238:0:13850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002248:0:13850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002255:0:13850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xc8b04d25 (ffff960d8e268300) 00010000:00010000:3.0:1785220586.002257:0:13850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220586.002260:0:13850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.002266:0:13850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a5fd0 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xc8b04d25 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13850 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.003259:0:13850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a5fc2 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdc7 expref: 477 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.003276:0:13850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220586.003279:0:13850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220586.003339:0:13850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a5fc2 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdc7 expref: 477 pid: 13850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.003358:0:13850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea68200, rc 0) 00000100:00100000:2.0:1785220586.003582:0:13850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_019:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53552:x1871938520027264:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 1764us (1976us total) trans 0 rc 301/301 00000100:00100000:2.0:1785220586.003593:0:13850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91810 00000100:00100000:1.0:1785220586.008713:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220586.008835:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220586.009036:0:13804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520027392 00000100:00100000:2.0:1785220586.009049:0:13804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520027392 00000100:00100000:2.0:1785220586.009136:0:13804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520027520 00000100:00100000:2.0:1785220586.009139:0:13804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520027520 00000100:00100000:2.0:1785220586.009171:0:13804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91811 00000100:00100000:2.0:1785220586.009181:0:13804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53479:x1871938520027392:12345-192.168.201.51@tcp:101:rm.0 00010000:00010000:2.0:1785220586.009215:0:13804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1785220586.009239:0:13804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93a00/0xf62499ab927a5fd7 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdce expref: -99 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009342:0:13804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009360:0:13804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009370:0:13804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009379:0:13804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009385:0:13804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009390:0:13804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009455:0:13804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92600/0xf62499ab927a5fec lrc: 3/1,0 mode: --/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009463:0:13804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92600/0xf62499ab927a5fec lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009469:0:13804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92600/0xf62499ab927a5fec lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1785220586.009547:0:13804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x98e:0x0]: returning size 4096 00000004:00200000:2.0:1785220586.009553:0:13804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1785220586.009572:0:13804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92600/0xf62499ab927a5fec lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009584:0:13804:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009594:0:13804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009600:0:13804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009607:0:13804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009613:0:13804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220586.009617:0:13804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009624:0:13804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009634:0:13804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:2.0:1785220586.009637:0:13804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220586.009641:0:13804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009647:0:13804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92800/0xf62499ab927a5fde lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009659:0:13804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009664:0:13804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009669:0:13804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009674:0:13804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220586.009677:0:13804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009683:0:13804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009702:0:13804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xa5ec40c9 (ffff960d8a305e00) 00010000:00010000:2.0:1785220586.009705:0:13804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220586.009708:0:13804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009713:0:13804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93600/0xf62499ab927a5fe5 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13804 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009738:0:13804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b93a00/0xf62499ab927a5fd7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdce expref: 478 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009749:0:13804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92600/0xf62499ab927a5fec lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdce expref: 477 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220586.009756:0:13804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x98e:0x0].0x0 (ffff960d8a305100) 00010000:00010000:2.0:1785220586.009759:0:13804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1785220586.009761:0:13804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b92600, rc 0) 00000100:00100000:1.0:1785220586.009906:0:13808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91812 00000100:00100000:1.0:1785220586.009918:0:13808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53824:x1871938520027520:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:1.0:1785220586.009933:0:13808:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220586.009948:0:13808:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a5ff3 lrc: 2/0,0 mode: --/CR res: [0x200000404:0x83d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdd5 expref: -99 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.010074:0:13808:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a5ff3 lrc: 1/0,0 mode: --/CR res: [0x200000404:0x83d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdd5 expref: 693 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.010108:0:13808:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000404:0x83d:0x0].0x0 (ffff960d8a305d00) 00010000:00010000:1.0:1785220586.010111:0:13808:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.010116:0:13808:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a5ff3 lrc: 0/0,0 mode: --/CR res: [0x200000404:0x83d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdd5 expref: 693 pid: 13808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.010130:0:13808:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496200, rc 0) 00000100:00100000:2.0:1785220586.010214:0:13804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5644b1e7-55e7-4421-a7e4-714e45fd656f+477:53479:x1871938520027392:12345-192.168.201.51@tcp:101:rm.0 Request processed in 1034us (1511us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220586.010222:0:13804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91811 00000100:00100000:1.0:1785220586.010649:0:13808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53824:x1871938520027520:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 734us (1813us total) trans 0 rc 301/301 00000100:00100000:1.0:1785220586.010657:0:13808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91812 00000100:00100000:2.0:1785220586.014608:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220586.014655:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220586.014724:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520027648 00000100:00100000:3.0:1785220586.014735:0:13847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520027904 00000100:00100000:0.0:1785220586.014736:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520027648 00000100:00100000:3.0:1785220586.014742:0:13847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520027904 00000100:00100000:0.0:1785220586.015076:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91813 00000100:00100000:0.0:1785220586.015111:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53824:x1871938520027648:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:0.0:1785220586.015132:0:6526:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1785220586.015155:0:6526:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3c00/0xf62499ab927a5ffa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fddc expref: -99 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220586.015378:0:13847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91814 00000100:00100000:3.0:1785220586.015387:0:13847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_018:5644b1e7-55e7-4421-a7e4-714e45fd656f+477:53479:x1871938520027904:12345-192.168.201.51@tcp:36:rm.0 00010000:00010000:3.0:1785220586.015427:0:13847:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220586.015448:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d90b92600/0xf62499ab927a5fec lrc: 0/0,0 mode: --/PR res: [0x240000402:0x98e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdce expref: 477 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015468:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x98e:0x0].0x0 (ffff960d8a305100) 00010000:00010000:3.0:1785220586.015473:0:13847:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1785220586.015488:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015500:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015509:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015519:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015526:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015532:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015544:0:6526:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015555:0:6526:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015561:0:6526:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015571:0:6526:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015579:0:6526:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015583:0:6526:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015603:0:6526:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015607:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015607:0:6526:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015618:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015625:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015644:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015651:0:13847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015663:0:13847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015664:0:6526:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015680:0:6526:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015691:0:6526:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220586.015696:0:6526:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015703:0:6526:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015713:0:6526:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:0.0:1785220586.015716:0:6526:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220586.015721:0:6526:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015722:0:13847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015736:0:6526:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2800/0xf62499ab927a600f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015738:0:13847:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.015758:0:13847:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015759:0:6526:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.015779:0:6526:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220586.015785:0:13847:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a2d3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_018:lustre-MDT0001-mdtlov_UUID:13847:1871938510815872:0@lo:101:mdt00_018.0 00000100:00100000:3.0:1785220586.015838:0:13847:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.015891:0:13847:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1785220586.015905:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510815872 00000100:00100000:1.0:1785220586.015911:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510815872 00000100:00100000:1.0:1785220586.015952:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12575 00000100:00100000:1.0:1785220586.015959:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:13847:x1871938510815872:12345-0@lo:101:mdt00_018.0 00010000:00010000:1.0:1785220586.015972:0:6531:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220586.015984:0:6531:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xf62499ab927a602b expref: -99 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.016002:0:6531:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8e4e1200/0xf62499ab927186e0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56561 expref: 693 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.016010:0:6531:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98bcc600/0xf62499ab92718a2f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56679 expref: 586 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.016016:0:6531:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 5/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 12 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.016039:0:6531:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960d98bcc600/0xf62499ab92718a2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56679 expref: 586 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.016061:0:6531:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98bcc600/0xf62499ab92718a2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56679 expref: 586 pid: 13805 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220586.016069:0:6531:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360a680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6531:1871938510816000:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220586.016155:0:6531:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960d8e4e1200/0xf62499ab927186e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56561 expref: 693 pid: 13812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.016161:0:6531:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8e4e1200/0xf62499ab927186e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56561 expref: 693 pid: 13812 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220586.016167:0:6531:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360b100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6531:1871938510816128:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220586.016185:0:6531:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8df01b40 going to sleep for 11 seconds 00010000:00010000:0.0:1785220586.016272:0:6526:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.016279:0:6526:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2a00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220586.016291:0:6526:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.016298:0:6526:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.016307:0:6526:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0xa5ec40c9 (ffff960d959b4300) 00010000:00010000:0.0:1785220586.016310:0:6526:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220586.016313:0:6526:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.016318:0:6526:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2a00/0xf62499ab927a6016 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6526 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.016360:0:6526:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3c00/0xf62499ab927a5ffa lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fddc expref: 693 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.016369:0:6526:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:0.0:1785220586.016371:0:6526:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1785220586.016375:0:6526:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3c00/0xf62499ab927a5ffa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fddc expref: 693 pid: 6526 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220586.016385:0:6526:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be3c00, rc 0) 00000100:00100000:0.0:1785220586.016413:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53824:x1871938520027648:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 1329us (1816us total) trans 0 rc 301/301 00000100:00100000:0.0:1785220586.016419:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91813 00000100:00100000:2.0:1785220586.022038:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220586.022079:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220586.022143:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220586.022324:0:13810:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520028288 00000100:00100000:1.0:1785220586.022334:0:13810:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520028288 00000100:00100000:1.0:1785220586.022391:0:13810:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91815 00000100:00100000:1.0:1785220586.022402:0:13810:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53972:x1871938520028288:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:1.0:1785220586.022422:0:13810:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220586.022443:0:13810:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a6039 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xafa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fde3 expref: -99 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.022606:0:13810:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a6039 lrc: 1/0,0 mode: --/CR res: [0x200000402:0xafa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fde3 expref: 693 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.022631:0:13810:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xafa:0x0].0x0 (ffff960d8a305d00) 00010000:00010000:1.0:1785220586.022634:0:13810:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.022639:0:13810:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a6039 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xafa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fde3 expref: 693 pid: 13810 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.022655:0:13810:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496200, rc 0) 00000100:00100000:1.0:1785220586.022716:0:13810:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:53972:x1871938520028288:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 316us (572us total) trans 0 rc 301/301 00000100:00100000:1.0:1785220586.022723:0:13810:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91815 00000100:00100000:1.0:1785220586.022780:0:6531:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360a680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6531:1871938510816000:192.168.201.51@tcp:104: 00000100:00100000:1.0:1785220586.022803:0:6531:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360b100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6531:1871938510816128:192.168.201.51@tcp:104: 00010000:00010000:1.0:1785220586.022831:0:6531:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 12 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.022839:0:6531:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff960d94c5f600) 00010000:00010000:1.0:1785220586.022841:0:6531:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220586.022845:0:6531:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 12 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.022855:0:6531:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 12 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.022861:0:6531:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496a00, rc 0) 00000100:00100000:1.0:1785220586.022904:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:13847:x1871938510815872:12345-0@lo:101:mdt00_018.0 Request processed in 6946us (7065us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220586.022911:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12575 00000100:00100000:1.0:1785220586.022947:0:13847:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a2d3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_018:lustre-MDT0001-mdtlov_UUID:13847:1871938510815872:0@lo:101:mdt00_018.0 00010000:00010000:1.0:1785220586.022963:0:13847:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 4/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.022973:0:13847:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 4/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220586.023051:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520028032 00000100:00100000:3.0:1785220586.023055:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520028032 00010000:00010000:3.0:1785220586.023078:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98bcc600/0xf62499ab92718a2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56679 expref: 587 pid: 13805 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220586.023126:0:14181:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520028160 00000100:00100000:3.0:1785220586.023129:0:14181:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520028160 00010000:00010000:3.0:1785220586.023141:0:14181:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d8e4e1200/0xf62499ab927186e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56561 expref: 692 pid: 13812 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220586.023159:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100837 00000100:00100000:3.0:1785220586.023166:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2404:x1871938520028032:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220586.023176:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220586.023182:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960d98bcc600/0xf62499ab92718a2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56679 expref: 587 pid: 13805 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023192:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960d98bcc600/0xf62499ab92718a2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56679 expref: 587 pid: 13805 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023206:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98bcc600/0xf62499ab92718a2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56679 expref: 587 pid: 13805 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023219:0:14181:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff960d94c5f600) 00010000:00010000:3.0:1785220586.023223:0:14181:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 11 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023231:0:14181:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 11 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023237:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220586.023258:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+586:2404:x1871938520028032:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 93us (1230us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.023265:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100837 00000100:00100000:3.0:1785220586.023282:0:14181:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 100838 00000100:00100000:3.0:1785220586.023288:0:14181:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921fb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:5644b1e7-55e7-4421-a7e4-714e45fd656f+692:2404:x1871938520028160:12345-192.168.201.51@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1785220586.023295:0:14181:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220586.023300:0:14181:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960d8e4e1200/0xf62499ab927186e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56561 expref: 692 pid: 13812 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023315:0:14181:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960d8e4e1200/0xf62499ab927186e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56561 expref: 692 pid: 13812 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023326:0:14181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8e4e1200/0xf62499ab927186e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.51@tcp remote: 0x384cc2c1b56561 expref: 692 pid: 13812 timeout: 1158 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023336:0:14181:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff960d94c5f600) 00010000:00010000:3.0:1785220586.023340:0:14181:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 11 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023348:0:14181:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 11 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023356:0:14181:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 11 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.023383:0:14181:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 11 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220586.023395:0:14181:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a2d0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000_UUID:14181:1871938510816256:0@lo:105: 00000100:00100000:3.0:1785220586.023426:0:14181:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.023452:0:14181:0:(client.c:2702:ptlrpc_set_wait()) set ffff960d8d38ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1785220586.024535:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510816256 00000100:00100000:1.0:1785220586.024542:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510816256 00000100:00100000:3.0:1785220586.024674:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2861 00000100:00100000:3.0:1785220586.024696:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:14181:x1871938510816256:12345-0@lo:105: 00010000:00010000:3.0:1785220586.024708:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 5/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.024723:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 5/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.024732:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 5/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.024740:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.024748:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.024889:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.024949:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff960d8ea68800) 00000100:00100000:3.0:1785220586.025011:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:14181:x1871938510816256:12345-0@lo:105: Request processed in 328us (1583us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.025017:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2861 00000100:00100000:0.0:1785220586.025123:0:14181:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a2d0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_009:lustre-MDT0000_UUID:14181:1871938510816256:0@lo:105: 00010000:00010000:0.0:1785220586.025141:0:14181:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1785220586.025186:0:14181:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_009:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:2404:x1871938520028160:12345-192.168.201.51@tcp:103:ldlm_bl.0 Request processed in 1896us (3105us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220586.025203:0:14181:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 100838 00010000:00010000:1.0:1785220586.025910:0:13847:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.025922:0:13847:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026035:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026043:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026049:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026054:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.026059:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026066:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026079:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x48:0x0].0x0 (ffff960d8e268300) 00010000:00010000:1.0:1785220586.026083:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220586.026432:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026439:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68600/0xf62499ab927a6024 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x48:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026463:0:13847:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026471:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026478:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026535:0:13847:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1785220586.026545:0:13847:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x3d:0x0] 00010000:00010000:1.0:1785220586.026566:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026572:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026578:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026583:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68c00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.026585:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026591:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026599:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x98e:0x0].0x0 (ffff960d8a305100) 00010000:00010000:1.0:1785220586.026602:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220586.026605:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 2/0,0 mode: --/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026676:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68c00/0xf62499ab927a601d lrc: 0/0,0 mode: --/EX res: [0x240000402:0x98e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026703:0:13847:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026709:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026715:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026721:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026726:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68200 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.026729:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026735:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026744:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220586.026746:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.026749:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026770:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68200/0xf62499ab927a6001 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026780:0:13847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026794:0:13847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026801:0:13847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026807:0:13847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.026810:0:13847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026816:0:13847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 4/0,0 mode: PW/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026824:0:13847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0xa5ec40c9 (ffff960d8a305e00) 00010000:00010000:1.0:1785220586.026826:0:13847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.026829:0:13847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 2/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.026835:0:13847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69800/0xf62499ab927a6008 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x1:0x0].0xa5ec40c9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13847 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1785220586.026949:0:13847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_018:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53479:x1871938520027904:12345-192.168.201.51@tcp:36:rm.0 Request processed in 11560us (12290us total) trans 0 rc -39/-39 00000100:00100000:1.0:1785220586.026957:0:13847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91814 00010000:00010000:3.0:1785220586.027311:0:13859:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1785220586.027318:0:13859:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.027326:0:13859:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc1a74ed0) 00010000:00010000:3.0:1785220586.027330:0:13859:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1785220586.027341:0:13859:0:(mdt_handler.c:3962:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.027370:0:13859:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.027377:0:13859:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1785220586.027408:0:13859:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 1/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.027415:0:13859:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff960d8ea68800/0xf62499ab927a602b lrc: 0/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xf62499ab927a6032 expref: -99 pid: 13847 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.027432:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220586.027433:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1785220586.027435:0:13859:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1785220586.027533:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8a2d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1871938510816384:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1785220586.027589:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.028495:0:13923:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510816384 00000100:00100000:3.0:1785220586.028505:0:13923:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510816384 00000100:00100000:3.0:1785220586.028568:0:13923:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14257 00000100:00100000:3.0:1785220586.028577:0:13923:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:lustre-MDT0001-mdtlov_UUID+12:3652:x1871938510816384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1785220586.028589:0:13923:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1785220586.028608:0:13923:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496a00/0xf62499ab927a6032 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xf62499ab927a602b expref: 12 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.028633:0:13923:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff960d94c5f600) 00010000:00010000:3.0:1785220586.028641:0:13923:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1785220586.028679:0:13923:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_008:lustre-MDT0001-mdtlov_UUID+11:3652:x1871938510816384:12345-0@lo:103:ldlm_bl.0 Request processed in 103us (1089us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.028697:0:13923:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14257 00000100:00100000:1.0:1785220586.028774:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8a2d2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1871938510816384:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1785220586.029038:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220586.029370:0:13918:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520028416 00000100:00100000:1.0:1785220586.029375:0:13918:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520028416 00000100:00100000:1.0:1785220586.029412:0:13918:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91816 00000100:00100000:1.0:1785220586.029419:0:13918:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d921f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_023:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:53972:x1871938520028416:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:1.0:1785220586.029435:0:13918:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220586.029449:0:13918:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496800/0xf62499ab927a6040 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdea expref: -99 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029538:0:13918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029551:0:13918:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029561:0:13918:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029575:0:13918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029581:0:13918:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029587:0:13918:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029703:0:13918:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029711:0:13918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029718:0:13918:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029725:0:13918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029731:0:13918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.029736:0:13918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029743:0:13918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029752:0:13918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:1.0:1785220586.029754:0:13918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.029871:0:13918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029878:0:13918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a6047 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029890:0:13918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029896:0:13918:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029903:0:13918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029909:0:13918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98497600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.029911:0:13918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029917:0:13918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029925:0:13918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0xdcc87686 (ffff960d8a305600) 00010000:00010000:1.0:1785220586.029927:0:13918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.029930:0:13918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029935:0:13918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98497600/0xf62499ab927a604e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029972:0:13918:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496800/0xf62499ab927a6040 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdea expref: 692 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029981:0:13918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:1.0:1785220586.029983:0:13918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1785220586.029987:0:13918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496800/0xf62499ab927a6040 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdea expref: 692 pid: 13918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.029994:0:13918:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496800, rc 0) 00000100:00100000:1.0:1785220586.030033:0:13918:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d921f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_023:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:53972:x1871938520028416:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 615us (997us total) trans 0 rc 301/301 00000100:00100000:1.0:1785220586.030039:0:13918:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91816 00000100:00100000:2.0:1785220586.032880:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220586.033382:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520028672 00000100:00100000:3.0:1785220586.033393:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520028672 00000100:00100000:3.0:1785220586.033444:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 25173 00000100:00100000:3.0:1785220586.033454:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960e83631850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5644b1e7-55e7-4421-a7e4-714e45fd656f+476:53614:x1871938520028672:12345-192.168.201.51@tcp:35:rm.0 00000100:00100000:0.0:1785220586.036048:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:2.0:1785220586.036822:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960e83631850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5644b1e7-55e7-4421-a7e4-714e45fd656f+477:53614:x1871938520028672:12345-192.168.201.51@tcp:35:rm.0 Request processed in 3365us (3947us total) trans 4294992696 rc 0/0 00000100:00100000:2.0:1785220586.036835:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 25173 00000100:00100000:3.0:1785220586.037068:0:13805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520028800 00000100:00100000:3.0:1785220586.037076:0:13805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520028800 00000100:00100000:3.0:1785220586.037168:0:13805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91817 00000100:00100000:3.0:1785220586.037178:0:13805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:54010:x1871938520028800:12345-192.168.201.51@tcp:101:mrename.0 00010000:00010000:3.0:1785220586.037199:0:13805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220586.037226:0:13805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a6055 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xafa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdf1 expref: -99 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.037414:0:13805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a6055 lrc: 1/0,0 mode: --/CR res: [0x200000402:0xafa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdf1 expref: 692 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.037428:0:13805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xafa:0x0].0x0 (ffff960d8e268d00) 00010000:00010000:3.0:1785220586.037431:0:13805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220586.037436:0:13805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a6055 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xafa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdf1 expref: 692 pid: 13805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.037452:0:13805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea69400, rc 0) 00000100:00100000:3.0:1785220586.037480:0:13805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:54010:x1871938520028800:12345-192.168.201.51@tcp:101:mrename.0 Request processed in 304us (1441us total) trans 0 rc 301/301 00000100:00100000:3.0:1785220586.037487:0:13805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91817 00000100:00100000:2.0:1785220586.042171:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220586.042300:0:13846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520029056 00000100:00100000:1.0:1785220586.042312:0:13846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520029056 00000100:00100000:1.0:1785220586.042375:0:13846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91818 00000100:00100000:1.0:1785220586.042386:0:13846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_017:5644b1e7-55e7-4421-a7e4-714e45fd656f+477:53614:x1871938520029056:12345-192.168.201.51@tcp:36:rm.0 00010000:00010000:1.0:1785220586.042444:0:13846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1785220586.042468:0:13846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9850fc00/0xf62499ab92726e57 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x936:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5c8fe expref: 477 pid: 13804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042484:0:13846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d9850e200/0xf62499ab9272698e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x936:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b5c6c7 expref: 476 pid: 12310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042492:0:13846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x936:0x0].0x0 (ffff960d91463400) 00010000:00010000:1.0:1785220586.042500:0:13846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220586.042517:0:13846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042528:0:13846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042535:0:13846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042543:0:13846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497600/0xf62499ab927a6063 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0x59d92b11 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042548:0:13846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497600/0xf62499ab927a6063 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042552:0:13846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497600/0xf62499ab927a6063 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042736:0:13846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x936:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042746:0:13846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.042753:0:13846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1785220586.043019:0:13846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x936:0x0]: returning size 4096 00010000:00010000:1.0:1785220586.043035:0:13846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043042:0:13846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043049:0:13846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043055:0:13846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.043059:0:13846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043066:0:13846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043074:0:13846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x936:0x0].0x0 (ffff960d8a305d00) 00010000:00010000:1.0:1785220586.043077:0:13846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220586.043080:0:13846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043747:0:13846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a606a lrc: 0/0,0 mode: --/EX res: [0x240000403:0x936:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043846:0:13846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043853:0:13846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043858:0:13846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043864:0:13846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043869:0:13846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.043872:0:13846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043878:0:13846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043889:0:13846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220586.043891:0:13846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.043894:0:13846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 3/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043900:0:13846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a605c lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1785220586.043909:0:13846:0:(mdt_handler.c:4478:mdt_save_lock()) ### save lock request 00000000511b7784 reply state 000000007060da20 transno 4294992697 ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497600/0xf62499ab927a6063 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043920:0:13846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98497600/0xf62499ab927a6063 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x1:0x0].0x59d92b11 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13846 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.043927:0:13846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x59d92b11 (ffff960d8a305600) 00010000:00010000:1.0:1785220586.043929:0:13846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1785220586.043931:0:13846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1785220586.044065:0:13846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9b008e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_017:5644b1e7-55e7-4421-a7e4-714e45fd656f+477:53614:x1871938520029056:12345-192.168.201.51@tcp:36:rm.0 Request processed in 1678us (1979us total) trans 4294992697 rc 0/0 00000100:00100000:1.0:1785220586.044073:0:13846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91818 00000100:00100000:0.0:1785220586.047795:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220586.052361:0:13814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520029184 00000100:00100000:3.0:1785220586.052377:0:13814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520029184 00000100:00100000:3.0:1785220586.052445:0:13814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91819 00000100:00100000:3.0:1785220586.052457:0:13814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+477:53560:x1871938520029184:12345-192.168.201.51@tcp:101:rm.0 00010000:00010000:3.0:1785220586.052482:0:13814:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220586.052505:0:13814:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69400/0xf62499ab927a6071 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdf8 expref: -99 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052605:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052620:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052633:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052646:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052654:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052661:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052945:0:13814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69600/0xf62499ab927a6086 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x975:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052954:0:13814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69600/0xf62499ab927a6086 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x975:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.052961:0:13814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69600/0xf62499ab927a6086 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x975:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1785220586.053001:0:13814:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x975:0x0]: returning size 4096 00000004:00200000:3.0:1785220586.053005:0:13814:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1785220586.053024:0:13814:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69600/0xf62499ab927a6086 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x975:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053037:0:13814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053043:0:13814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053049:0:13814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053193:0:13814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053201:0:13814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220586.053205:0:13814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053212:0:13814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053223:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:3.0:1785220586.053225:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220586.053230:0:13814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 3/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053236:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68800/0xf62499ab927a6078 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053254:0:13814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053261:0:13814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053266:0:13814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053272:0:13814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68a00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220586.053274:0:13814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053290:0:13814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053298:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x1d91fb2c (ffff960d8e268d00) 00010000:00010000:3.0:1785220586.053301:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220586.053305:0:13814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 2/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053309:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea68a00/0xf62499ab927a607f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13814 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053344:0:13814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69400/0xf62499ab927a6071 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdf8 expref: 479 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053359:0:13814:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69600/0xf62499ab927a6086 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x975:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdf8 expref: 478 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220586.053371:0:13814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x975:0x0].0x0 (ffff960d8e268400) 00010000:00010000:3.0:1785220586.053373:0:13814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1785220586.053375:0:13814:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea69600, rc 0) 00000100:00100000:2.0:1785220586.053563:0:13814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_015:5644b1e7-55e7-4421-a7e4-714e45fd656f+478:53560:x1871938520029184:12345-192.168.201.51@tcp:101:rm.0 Request processed in 1106us (5776us total) trans 0 rc 0/0 00000100:00100000:2.0:1785220586.053572:0:13814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91819 00000100:00100000:0.0:1785220586.057656:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1785220586.057764:0:13872:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520029440 00000100:00100000:1.0:1785220586.057775:0:13872:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520029440 00000100:00100000:1.0:1785220586.057925:0:13872:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91820 00000100:00100000:1.0:1785220586.057937:0:13872:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:5644b1e7-55e7-4421-a7e4-714e45fd656f+478:53560:x1871938520029440:12345-192.168.201.51@tcp:36:rm.0 00010000:00010000:1.0:1785220586.057989:0:13872:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1785220586.058016:0:13872:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d8ea69600/0xf62499ab927a6086 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x975:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdf8 expref: 478 pid: 13814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058035:0:13872:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x975:0x0].0x0 (ffff960d8e268400) 00010000:00010000:1.0:1785220586.058049:0:13872:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1785220586.058081:0:13872:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 3/0,1 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13872 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058403:0:13872:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058413:0:13872:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058428:0:13872:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a6094 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13872 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058434:0:13872:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a6094 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058441:0:13872:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a6094 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058520:0:13872:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x975:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13872 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058528:0:13872:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058534:0:13872:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1785220586.058833:0:13872:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x975:0x0]: returning size 4096 00010000:00010000:1.0:1785220586.058850:0:13872:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058859:0:13872:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058865:0:13872:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058871:0:13872:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.058876:0:13872:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058884:0:13872:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058893:0:13872:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x975:0x0].0x0 (ffff960d8a305f00) 00010000:00010000:1.0:1785220586.058895:0:13872:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220586.058899:0:13872:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 2/0,0 mode: --/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058905:0:13872:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496400/0xf62499ab927a609b lrc: 0/0,0 mode: --/EX res: [0x240000403:0x975:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058924:0:13872:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058930:0:13872:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058936:0:13872:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058942:0:13872:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058947:0:13872:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496c00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220586.058950:0:13872:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058956:0:13872:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058963:0:13872:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff960d86fd2c00) 00010000:00010000:1.0:1785220586.058965:0:13872:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220586.058969:0:13872:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 3/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058974:0:13872:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496c00/0xf62499ab927a608d lrc: 0/0,0 mode: --/CW res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1785220586.058983:0:13872:0:(mdt_handler.c:4478:mdt_save_lock()) ### save lock request 000000004db174b9 reply state 000000009a1e29f0 transno 4294992698 ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a6094 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058993:0:13872:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff960d98496800/0xf62499ab927a6094 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x1:0x0].0x1d91fb2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13872 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220586.058999:0:13872:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x1d91fb2c (ffff960d8a305d00) 00010000:00010000:1.0:1785220586.059001:0:13872:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1785220586.059003:0:13872:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1785220586.060794:0:13872:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_022:5644b1e7-55e7-4421-a7e4-714e45fd656f+479:53560:x1871938520029440:12345-192.168.201.51@tcp:36:rm.0 Request processed in 2851us (3140us total) trans 4294992698 rc 0/0 00000100:00100000:1.0:1785220586.060805:0:13872:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91820 00000100:00100000:1.0:1785220586.065328:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220586.065458:0:13920:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520029696 00000100:00100000:3.0:1785220586.065469:0:13920:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520029696 00000100:00100000:3.0:1785220586.065534:0:13920:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 91821 00000100:00100000:3.0:1785220586.065546:0:13920:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9360a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_025:5644b1e7-55e7-4421-a7e4-714e45fd656f+479:53922:x1871938520029696:12345-192.168.201.51@tcp:36:mkdir.0 00000100:00100000:3.0:1785220586.065721:0:13920:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9360a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_025:5644b1e7-55e7-4421-a7e4-714e45fd656f+479:53922:x1871938520029696:12345-192.168.201.51@tcp:36:mkdir.0 Request processed in 178us (402us total) trans 0 rc -17/-17 00000100:00100000:3.0:1785220586.065728:0:13920:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 91821 00000100:00100000:1.0:1785220586.217356:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1785220586.217382:0:3652:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d93608380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1871938510816512:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1785220586.217585:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.217724:0:13309:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510816512 00000100:00100000:3.0:1785220586.217734:0:13309:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510816512 00000100:00100000:3.0:1785220586.217804:0:13309:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:3.0:1785220586.217814:0:13309:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d93609880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1871938510816512:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1785220586.217837:0:13309:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 143360 granted 970222400 pending 0 free 3903991808 avail 3672469504 00000100:00100000:2.0:1785220586.217845:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d984e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1871938510816768:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.217875:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.217879:0:13309:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d93609880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1871938510816512:12345-0@lo:13:osp-pre-0-1.0 Request processed in 68us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.217886:0:13309:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:2.0:1785220586.217901:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1871938510817280:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.217910:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510817024:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.217923:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.217933:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220586.217936:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1871938510817792:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.217946:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1871938510817536:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.217956:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.217962:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.217973:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1785220586.217973:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1785220586.217978:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1871938510816896:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.217978:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d984e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4add7b40-ab9d-45be-a6c9-eb07929d24cd:3655:1871938510816640:0@lo:400:kworker.0 00000100:00100000:0.0:1785220586.218075:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510816768 00000100:00100000:0.0:1785220586.218080:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510816768 00000100:00100000:0.0:1785220586.218131:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817280 00000100:00100000:0.0:1785220586.218134:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817280 00000100:00100000:0.0:1785220586.218155:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817536 00000100:00100000:0.0:1785220586.218158:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817536 00000100:00100000:0.0:1785220586.218177:0:13921:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1354 00000100:00100000:0.0:1785220586.218184:0:13921:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8e5b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1871938510816768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1785220586.218248:0:12310:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1355 00000100:00100000:1.0:1785220586.218256:0:12310:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d96799f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1871938510817280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218313:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.218334:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510817408:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218357:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.218373:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510817920:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218390:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.218407:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9a707480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1871938510818048:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1785220586.218425:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1785220586.218453:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817792 00000100:00100000:3.0:1785220586.218457:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817792 00000100:00100000:3.0:1785220586.218479:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817408 00000100:00100000:3.0:1785220586.218482:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817408 00000100:00100000:3.0:1785220586.218496:0:14259:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817920 00000100:00100000:3.0:1785220586.218499:0:14259:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817920 00000100:00100000:3.0:1785220586.218516:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:3.0:1785220586.218521:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9679b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0001-mdtlov_UUID+5:3655:x1871938510817792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218548:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9679b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0001-mdtlov_UUID+5:3655:x1871938510817792:12345-0@lo:400:kworker.0 Request processed in 28us (591us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.218554:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:3.0:1785220586.218566:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1785220586.218572:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+206:3653:x1871938510817408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218596:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+206:3653:x1871938510817408:12345-0@lo:400:kworker.0 Request processed in 25us (238us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.218602:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1785220586.218612:0:14259:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1785220586.218617:0:14259:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+66:3653:x1871938510817920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218639:0:14259:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_082:lustre-MDT0000-mdtlov_UUID+66:3653:x1871938510817920:12345-0@lo:400:kworker.0 Request processed in 23us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.218644:0:14259:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1785220586.218660:0:13811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510816896 00000100:00100000:3.0:1785220586.218663:0:13811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510816896 00000100:00100000:3.0:1785220586.218694:0:13811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1356 00000100:00100000:3.0:1785220586.218700:0:13811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1871938510817536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218726:0:13811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1871938510817536:12345-0@lo:400:kworker.0 Request processed in 27us (763us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.218730:0:13811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1356 00000100:00100000:3.0:1785220586.218741:0:13811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1357 00000100:00100000:3.0:1785220586.218746:0:13811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1871938510816896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218890:0:13811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1871938510816896:12345-0@lo:400:kworker.0 Request processed in 143us (576us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.218896:0:13811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1357 00000100:00100000:3.0:1785220586.218931:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1871938510816896:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218941:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510817408:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218950:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510817920:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.218959:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1871938510817536:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.219853:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510818048 00000100:00100000:3.0:1785220586.219860:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510818048 00000100:00100000:3.0:1785220586.219897:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:3.0:1785220586.219906:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1871938510818048:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1785220586.219925:0:8444:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 286720 granted 582591936 pending 57344 free 3891458048 avail 3659935744 00000100:00100000:3.0:1785220586.219971:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1871938510818048:12345-0@lo:13:osp-pre-1-1.0 Request processed in 67us (1544us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220586.219978:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00080000:3.0:1785220586.219998:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1785220586.220002:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1785220586.220078:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1785220586.220082:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1785220586.220348:0:3653:0:(client.c:3112:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4295069367 gen 2 00000004:00080000:3.0:1785220586.220354:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc0380, transno 4295068367 00000004:00080000:3.0:1785220586.220364:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc3100, transno 4295068369 00000004:00080000:3.0:1785220586.220368:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc2300, transno 4295068371 00000004:00080000:3.0:1785220586.220371:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cfb80, transno 4295068374 00000004:00080000:3.0:1785220586.220373:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cce00, transno 4295068376 00000004:00080000:3.0:1785220586.220376:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cd180, transno 4295068377 00000004:00080000:3.0:1785220586.220378:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d792a00, transno 4295068380 00000004:00080000:3.0:1785220586.220382:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfdc00, transno 4295068382 00000004:00080000:3.0:1785220586.220384:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98485880, transno 4295068383 00000004:00080000:3.0:1785220586.220387:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc1880, transno 4295068386 00000004:00080000:3.0:1785220586.220389:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00f100, transno 4295068388 00000004:00080000:3.0:1785220586.220391:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986f100, transno 4295068389 00000004:00080000:3.0:1785220586.220394:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00f480, transno 4295068390 00000004:00080000:3.0:1785220586.220397:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca9c00, transno 4295068393 00000004:00080000:3.0:1785220586.220399:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986dc00, transno 4295068394 00000004:00080000:3.0:1785220586.220402:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca8a80, transno 4295068397 00000004:00080000:3.0:1785220586.220404:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c62a00, transno 4295068399 00000004:00080000:3.0:1785220586.220407:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e9f80, transno 4295068405 00000004:00080000:3.0:1785220586.220409:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e8a80, transno 4295068406 00000004:00080000:3.0:1785220586.220412:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979ebb80, transno 4295068407 00000004:00080000:3.0:1785220586.220414:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bfb80, transno 4295068408 00000004:00080000:3.0:1785220586.220417:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979baa00, transno 4295068409 00000004:00080000:3.0:1785220586.220420:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b8e00, transno 4295068410 00000004:00080000:3.0:1785220586.220423:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b8a80, transno 4295068411 00000004:00080000:3.0:1785220586.220426:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979bb800, transno 4295068416 00000004:00080000:3.0:1785220586.220428:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bc380, transno 4295068417 00000004:00080000:3.0:1785220586.220430:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19ca80, transno 4295068420 00000004:00080000:3.0:1785220586.220433:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996df80, transno 4295068421 00000004:00080000:3.0:1785220586.220436:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19ea00, transno 4295068425 00000004:00080000:3.0:1785220586.220439:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bdc00, transno 4295068426 00000004:00080000:3.0:1785220586.220441:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19ce00, transno 4295068428 00000004:00080000:3.0:1785220586.220443:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996fb80, transno 4295068430 00000004:00080000:3.0:1785220586.220446:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f68e00, transno 4295068432 00000004:00080000:3.0:1785220586.220449:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996ca80, transno 4295068434 00000004:00080000:3.0:1785220586.220452:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996ea00, transno 4295068436 00000004:00080000:3.0:1785220586.220454:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d791c00, transno 4295068439 00000004:00080000:3.0:1785220586.220457:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d790e00, transno 4295068444 00000004:00080000:3.0:1785220586.220460:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236aa00, transno 4295068445 00000004:00080000:3.0:1785220586.220462:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d791880, transno 4295068446 00000004:00080000:3.0:1785220586.220464:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d793480, transno 4295068449 00000004:00080000:3.0:1785220586.220467:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d792300, transno 4295068450 00000004:00080000:3.0:1785220586.220469:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d791f80, transno 4295068451 00000004:00080000:3.0:1785220586.220472:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92369c00, transno 4295068454 00000004:00080000:3.0:1785220586.220474:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00e680, transno 4295068455 00000004:00080000:3.0:1785220586.220477:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236b800, transno 4295068457 00000004:00080000:3.0:1785220586.220479:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92368700, transno 4295068458 00000004:00080000:3.0:1785220586.220481:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236b480, transno 4295068463 00000004:00080000:3.0:1785220586.220484:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00fb80, transno 4295068464 00000004:00080000:3.0:1785220586.220486:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d793b80, transno 4295068466 00000004:00080000:3.0:1785220586.220489:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00df80, transno 4295068468 00000004:00080000:3.0:1785220586.220491:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995e300, transno 4295068469 00000004:00080000:3.0:1785220586.220494:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684c000, transno 4295068472 00000004:00080000:3.0:1785220586.220497:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5d500, transno 4295068475 00000004:00080000:3.0:1785220586.220500:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98412d80, transno 4295068476 00000004:00080000:3.0:1785220586.220503:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9996c700, transno 4295068477 00000004:00080000:3.0:1785220586.220505:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98413b80, transno 4295068480 00000004:00080000:3.0:1785220586.220507:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98410380, transno 4295068481 00000004:00080000:3.0:1785220586.220510:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98413480, transno 4295068483 00000004:00080000:3.0:1785220586.220513:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98411880, transno 4295068484 00000004:00080000:3.0:1785220586.220515:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f1f80, transno 4295068487 00000004:00080000:3.0:1785220586.220517:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f2680, transno 4295068488 00000004:00080000:3.0:1785220586.220520:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c63480, transno 4295068489 00000004:00080000:3.0:1785220586.220523:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f0a80, transno 4295068490 00000004:00080000:3.0:1785220586.220526:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6f100, transno 4295068496 00000004:00080000:3.0:1785220586.220528:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97442a00, transno 4295068498 00000004:00080000:3.0:1785220586.220531:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e9880, transno 4295068499 00000004:00080000:3.0:1785220586.220533:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97440380, transno 4295068502 00000004:00080000:3.0:1785220586.220536:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe9500, transno 4295068503 00000004:00080000:3.0:1785220586.220539:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97443b80, transno 4295068506 00000004:00080000:3.0:1785220586.220541:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97441180, transno 4295068507 00000004:00080000:3.0:1785220586.220544:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97442d80, transno 4295068509 00000004:00080000:3.0:1785220586.220546:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8ea00, transno 4295068510 00000004:00080000:3.0:1785220586.220549:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f2d80, transno 4295068513 00000004:00080000:3.0:1785220586.220552:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977a8a80, transno 4295068514 00000004:00080000:3.0:1785220586.220555:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f3b80, transno 4295068518 00000004:00080000:3.0:1785220586.220557:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c2bb80, transno 4295068521 00000004:00080000:3.0:1785220586.220560:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c28700, transno 4295068522 00000004:00080000:3.0:1785220586.220562:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735ca80, transno 4295068526 00000004:00080000:3.0:1785220586.220565:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977aa680, transno 4295068527 00000004:00080000:3.0:1785220586.220568:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986eca80, transno 4295068530 00000004:00080000:3.0:1785220586.220570:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977aa300, transno 4295068531 00000004:00080000:3.0:1785220586.220573:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794c000, transno 4295068532 00000004:00080000:3.0:1785220586.220576:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977a9500, transno 4295068533 00000004:00080000:3.0:1785220586.220579:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856ea00, transno 4295068536 00000004:00080000:3.0:1785220586.220582:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908b8700, transno 4295068539 00000004:00080000:3.0:1785220586.220584:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92167b80, transno 4295068540 00000004:00080000:3.0:1785220586.220587:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8ce00, transno 4295068543 00000004:00080000:3.0:1785220586.220589:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e5b4700, transno 4295068545 00000004:00080000:3.0:1785220586.220592:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908bb100, transno 4295068546 00000004:00080000:3.0:1785220586.220594:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856d880, transno 4295068547 00000004:00080000:3.0:1785220586.220597:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbd500, transno 4295068549 00000004:00080000:3.0:1785220586.220600:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8ca80, transno 4295068550 00000004:00080000:3.0:1785220586.220602:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbea00, transno 4295068556 00000004:00080000:3.0:1785220586.220605:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbc380, transno 4295068559 00000004:00080000:3.0:1785220586.220608:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbbb800, transno 4295068560 00000004:00080000:3.0:1785220586.220611:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f2300, transno 4295068562 00000004:00080000:3.0:1785220586.220613:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb8000, transno 4295068564 00000004:00080000:3.0:1785220586.220616:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbb9880, transno 4295068565 00000004:00080000:3.0:1785220586.220618:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de7480, transno 4295068566 00000004:00080000:3.0:1785220586.220621:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfbbad80, transno 4295068569 00000004:00080000:3.0:1785220586.220624:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa64e00, transno 4295068572 00000004:00080000:3.0:1785220586.220626:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa65c00, transno 4295068575 00000004:00080000:3.0:1785220586.220629:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa66300, transno 4295068576 00000004:00080000:3.0:1785220586.220631:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e5b5c00, transno 4295068577 00000004:00080000:3.0:1785220586.220634:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c2b100, transno 4295068578 00000004:00080000:3.0:1785220586.220637:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c28e00, transno 4295068580 00000004:00080000:3.0:1785220586.220640:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e5b7100, transno 4295068581 00000004:00080000:3.0:1785220586.220642:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa66d80, transno 4295068582 00000004:00080000:3.0:1785220586.220645:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b34b100, transno 4295068585 00000004:00080000:3.0:1785220586.220648:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b348700, transno 4295068590 00000004:00080000:3.0:1785220586.220650:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b348380, transno 4295068591 00000004:00080000:3.0:1785220586.220653:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b34aa00, transno 4295068595 00000004:00080000:3.0:1785220586.220655:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236bb80, transno 4295068596 00000004:00080000:3.0:1785220586.220658:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d1500, transno 4295068597 00000004:00080000:3.0:1785220586.220661:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d3100, transno 4295068598 00000004:00080000:3.0:1785220586.220663:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d0e00, transno 4295068599 00000004:00080000:3.0:1785220586.220666:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b349500, transno 4295068600 00000004:00080000:3.0:1785220586.220668:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94deed80, transno 4295068606 00000004:00080000:3.0:1785220586.220671:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96418e00, transno 4295068607 00000004:00080000:3.0:1785220586.220673:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e84000, transno 4295068609 00000004:00080000:3.0:1785220586.220676:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e85880, transno 4295068611 00000004:00080000:3.0:1785220586.220678:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae21f80, transno 4295068612 00000004:00080000:3.0:1785220586.220693:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae23100, transno 4295068613 00000004:00080000:3.0:1785220586.220697:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae22a00, transno 4295068615 00000004:00080000:3.0:1785220586.220699:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ee300, transno 4295068616 00000004:00080000:3.0:1785220586.220701:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae20a80, transno 4295068622 00000004:00080000:3.0:1785220586.220703:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99374a80, transno 4295068623 00000004:00080000:3.0:1785220586.220706:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99375500, transno 4295068624 00000004:00080000:3.0:1785220586.220708:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99377480, transno 4295068625 00000004:00080000:3.0:1785220586.220710:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794f100, transno 4295068626 00000004:00080000:3.0:1785220586.220713:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99377b80, transno 4295068627 00000004:00080000:3.0:1785220586.220715:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99374000, transno 4295068632 00000004:00080000:3.0:1785220586.220717:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56ca80, transno 4295068635 00000004:00080000:3.0:1785220586.220720:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b98a80, transno 4295068636 00000004:00080000:3.0:1785220586.220723:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56e680, transno 4295068638 00000004:00080000:3.0:1785220586.220725:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986edf80, transno 4295068641 00000004:00080000:3.0:1785220586.220728:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99374e00, transno 4295068642 00000004:00080000:3.0:1785220586.220730:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56d180, transno 4295068643 00000004:00080000:3.0:1785220586.220733:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c549f80, transno 4295068644 00000004:00080000:3.0:1785220586.220735:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94dec700, transno 4295068648 00000004:00080000:3.0:1785220586.220738:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91601880, transno 4295068651 00000004:00080000:3.0:1785220586.220740:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91600380, transno 4295068652 00000004:00080000:3.0:1785220586.220743:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91600e00, transno 4295068655 00000004:00080000:3.0:1785220586.220746:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b9b480, transno 4295068656 00000004:00080000:3.0:1785220586.220748:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91602680, transno 4295068657 00000004:00080000:3.0:1785220586.220750:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b99880, transno 4295068661 00000004:00080000:3.0:1785220586.221471:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b98e00, transno 4295068664 00000004:00080000:3.0:1785220586.221476:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b98380, transno 4295068665 00000004:00080000:3.0:1785220586.221478:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb79c00, transno 4295068667 00000004:00080000:3.0:1785220586.221481:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb78a80, transno 4295068669 00000004:00080000:3.0:1785220586.221483:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dc700, transno 4295068670 00000004:00080000:3.0:1785220586.221486:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb7b800, transno 4295068672 00000004:00080000:3.0:1785220586.221488:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dea00, transno 4295068673 00000004:00080000:3.0:1785220586.221490:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb7b100, transno 4295068676 00000004:00080000:3.0:1785220586.221493:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb78000, transno 4295068679 00000004:00080000:3.0:1785220586.221495:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb7ad80, transno 4295068682 00000004:00080000:3.0:1785220586.221498:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99935500, transno 4295068683 00000004:00080000:3.0:1785220586.221500:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fe300, transno 4295068686 00000004:00080000:3.0:1785220586.221503:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e71180, transno 4295068687 00000004:00080000:3.0:1785220586.221505:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e71880, transno 4295068690 00000004:00080000:3.0:1785220586.221507:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc3100, transno 4295068691 00000004:00080000:3.0:1785220586.221510:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc0e00, transno 4295068692 00000004:00080000:3.0:1785220586.221512:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e71f80, transno 4295068693 00000004:00080000:3.0:1785220586.221515:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc1f80, transno 4295068700 00000004:00080000:3.0:1785220586.221517:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99025f80, transno 4295068702 00000004:00080000:3.0:1785220586.221520:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efa680, transno 4295068704 00000004:00080000:3.0:1785220586.221522:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a8e00, transno 4295068706 00000004:00080000:3.0:1785220586.221525:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99934700, transno 4295068709 00000004:00080000:3.0:1785220586.221527:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99026d80, transno 4295068710 00000004:00080000:3.0:1785220586.221530:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99027b80, transno 4295068711 00000004:00080000:3.0:1785220586.221532:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a8a80, transno 4295068713 00000004:00080000:3.0:1785220586.221534:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a9c00, transno 4295068714 00000004:00080000:3.0:1785220586.221537:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d1c00, transno 4295068717 00000004:00080000:3.0:1785220586.221540:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d3b80, transno 4295068719 00000004:00080000:3.0:1785220586.221543:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef3800, transno 4295068723 00000004:00080000:3.0:1785220586.221545:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef3100, transno 4295068724 00000004:00080000:3.0:1785220586.221547:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef1c00, transno 4295068726 00000004:00080000:3.0:1785220586.221550:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d1f80, transno 4295068729 00000004:00080000:3.0:1785220586.221552:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d0a80, transno 4295068730 00000004:00080000:3.0:1785220586.221554:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906aa680, transno 4295068732 00000004:00080000:3.0:1785220586.221557:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef2a00, transno 4295068733 00000004:00080000:3.0:1785220586.221559:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d638380, transno 4295068734 00000004:00080000:3.0:1785220586.221562:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1df80, transno 4295068740 00000004:00080000:3.0:1785220586.221564:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96419880, transno 4295068741 00000004:00080000:3.0:1785220586.221567:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e3100, transno 4295068743 00000004:00080000:3.0:1785220586.221570:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e3b80, transno 4295068745 00000004:00080000:3.0:1785220586.221572:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0eca80, transno 4295068746 00000004:00080000:3.0:1785220586.221575:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964ded80, transno 4295068747 00000004:00080000:3.0:1785220586.221578:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0eea00, transno 4295068748 00000004:00080000:3.0:1785220586.221580:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ec380, transno 4295068750 00000004:00080000:3.0:1785220586.221583:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ef100, transno 4295068754 00000004:00080000:3.0:1785220586.221585:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dca80, transno 4295068757 00000004:00080000:3.0:1785220586.221587:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ece00, transno 4295068758 00000004:00080000:3.0:1785220586.221590:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bdf80, transno 4295068761 00000004:00080000:3.0:1785220586.221593:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93573480, transno 4295068762 00000004:00080000:3.0:1785220586.221596:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93571c00, transno 4295068763 00000004:00080000:3.0:1785220586.221598:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bc700, transno 4295068766 00000004:00080000:3.0:1785220586.221601:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964dd180, transno 4295068770 00000004:00080000:3.0:1785220586.221603:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7f800, transno 4295068771 00000004:00080000:3.0:1785220586.221606:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999be680, transno 4295068772 00000004:00080000:3.0:1785220586.221608:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95449f80, transno 4295068774 00000004:00080000:3.0:1785220586.221610:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7d500, transno 4295068775 00000004:00080000:3.0:1785220586.221612:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95448000, transno 4295068777 00000004:00080000:3.0:1785220586.221615:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544a680, transno 4295068778 00000004:00080000:3.0:1785220586.221617:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95448380, transno 4295068780 00000004:00080000:3.0:1785220586.221620:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7e680, transno 4295068785 00000004:00080000:3.0:1785220586.221622:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7d880, transno 4295068786 00000004:00080000:3.0:1785220586.221625:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bfb80, transno 4295068787 00000004:00080000:3.0:1785220586.221627:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986edc00, transno 4295068788 00000004:00080000:3.0:1785220586.221630:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d1500, transno 4295068789 00000004:00080000:3.0:1785220586.221632:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7df80, transno 4295068792 00000004:00080000:3.0:1785220586.221635:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480d500, transno 4295068797 00000004:00080000:3.0:1785220586.221637:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef0380, transno 4295068799 00000004:00080000:3.0:1785220586.221640:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93571f80, transno 4295068800 00000004:00080000:3.0:1785220586.221643:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef7800, transno 4295068804 00000004:00080000:3.0:1785220586.221646:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faaa300, transno 4295068805 00000004:00080000:3.0:1785220586.221648:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91170e00, transno 4295068807 00000004:00080000:3.0:1785220586.221650:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0c8380, transno 4295068808 00000004:00080000:3.0:1785220586.221653:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91170000, transno 4295068809 00000004:00080000:3.0:1785220586.221655:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc34e00, transno 4295068810 00000004:00080000:3.0:1785220586.221657:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc34700, transno 4295068812 00000004:00080000:3.0:1785220586.221660:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb84380, transno 4295068815 00000004:00080000:3.0:1785220586.221663:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb84e00, transno 4295068819 00000004:00080000:3.0:1785220586.221665:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc34a80, transno 4295068820 00000004:00080000:3.0:1785220586.221668:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90406300, transno 4295068821 00000004:00080000:3.0:1785220586.221670:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676bb80, transno 4295068823 00000004:00080000:3.0:1785220586.221673:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96768700, transno 4295068825 00000004:00080000:3.0:1785220586.221675:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af7100, transno 4295068826 00000004:00080000:3.0:1785220586.221678:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676a300, transno 4295068828 00000004:00080000:3.0:1785220586.221692:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af6300, transno 4295068830 00000004:00080000:3.0:1785220586.221696:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95538000, transno 4295068833 00000004:00080000:3.0:1785220586.221698:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af6d80, transno 4295068835 00000004:00080000:3.0:1785220586.221700:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af7480, transno 4295068838 00000004:00080000:3.0:1785220586.221703:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90404000, transno 4295068839 00000004:00080000:3.0:1785220586.221706:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af5f80, transno 4295068840 00000004:00080000:3.0:1785220586.221708:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c168000, transno 4295068842 00000004:00080000:3.0:1785220586.221710:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16a300, transno 4295068843 00000004:00080000:3.0:1785220586.221712:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16aa00, transno 4295068844 00000004:00080000:3.0:1785220586.221715:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af5500, transno 4295068848 00000004:00080000:3.0:1785220586.221717:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c169880, transno 4295068850 00000004:00080000:3.0:1785220586.221720:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfbb80, transno 4295068853 00000004:00080000:3.0:1785220586.221722:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c973480, transno 4295068854 00000004:00080000:3.0:1785220586.221725:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e2680, transno 4295068855 00000004:00080000:3.0:1785220586.221727:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c971500, transno 4295068857 00000004:00080000:3.0:1785220586.221730:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b17800, transno 4295068860 00000004:00080000:3.0:1785220586.221732:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98624380, transno 4295068861 00000004:00080000:3.0:1785220586.221735:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c18380, transno 4295068865 00000004:00080000:3.0:1785220586.221737:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73bb80, transno 4295068866 00000004:00080000:3.0:1785220586.221740:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c19500, transno 4295068867 00000004:00080000:3.0:1785220586.221742:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b5500, transno 4295068870 00000004:00080000:3.0:1785220586.221744:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b5180, transno 4295068871 00000004:00080000:3.0:1785220586.221747:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa4000, transno 4295068873 00000004:00080000:3.0:1785220586.221749:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa5c00, transno 4295068874 00000004:00080000:3.0:1785220586.222849:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e1180, transno 4295068878 00000004:00080000:3.0:1785220586.222859:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e3100, transno 4295068880 00000004:00080000:3.0:1785220586.222861:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf9c00, transno 4295068882 00000004:00080000:3.0:1785220586.222864:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb1c00, transno 4295068884 00000004:00080000:3.0:1785220586.222866:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb2d80, transno 4295068886 00000004:00080000:3.0:1785220586.222869:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98624e00, transno 4295068888 00000004:00080000:3.0:1785220586.222871:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afb480, transno 4295068889 00000004:00080000:3.0:1785220586.222873:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98624a80, transno 4295068890 00000004:00080000:3.0:1785220586.222876:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b6300, transno 4295068895 00000004:00080000:3.0:1785220586.222879:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93740a80, transno 4295068897 00000004:00080000:3.0:1785220586.222883:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93742a00, transno 4295068898 00000004:00080000:3.0:1785220586.222885:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91505500, transno 4295068900 00000004:00080000:3.0:1785220586.222887:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91505c00, transno 4295068902 00000004:00080000:3.0:1785220586.222890:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364ad80, transno 4295068907 00000004:00080000:3.0:1785220586.222892:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93743480, transno 4295068908 00000004:00080000:3.0:1785220586.222894:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93741880, transno 4295068909 00000004:00080000:3.0:1785220586.222897:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93648000, transno 4295068910 00000004:00080000:3.0:1785220586.222899:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95453b80, transno 4295068913 00000004:00080000:3.0:1785220586.222902:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362b480, transno 4295068916 00000004:00080000:3.0:1785220586.222904:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfc000, transno 4295068919 00000004:00080000:3.0:1785220586.222906:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1e680, transno 4295068922 00000004:00080000:3.0:1785220586.222909:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1fb80, transno 4295068924 00000004:00080000:3.0:1785220586.222911:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95451880, transno 4295068925 00000004:00080000:3.0:1785220586.222913:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dffb80, transno 4295068926 00000004:00080000:3.0:1785220586.222916:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1e300, transno 4295068927 00000004:00080000:3.0:1785220586.222919:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93631f80, transno 4295068928 00000004:00080000:3.0:1785220586.222921:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93633b80, transno 4295068932 00000004:00080000:3.0:1785220586.222923:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98484380, transno 4295068933 00000004:00080000:3.0:1785220586.222925:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cca80, transno 4295068939 00000004:00080000:3.0:1785220586.222928:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cc000, transno 4295068940 00000004:00080000:3.0:1785220586.222930:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cf100, transno 4295068943 00000004:00080000:3.0:1785220586.222933:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cdc00, transno 4295068945 00000004:00080000:3.0:1785220586.222935:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cc700, transno 4295068946 00000004:00080000:3.0:1785220586.222937:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb0e00, transno 4295068947 00000004:00080000:3.0:1785220586.222940:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfd180, transno 4295068948 00000004:00080000:3.0:1785220586.222942:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb1880, transno 4295068949 00000004:00080000:3.0:1785220586.222945:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfed80, transno 4295068954 00000004:00080000:3.0:1785220586.222947:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958ced80, transno 4295068956 00000004:00080000:3.0:1785220586.222949:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfe680, transno 4295068958 00000004:00080000:3.0:1785220586.222951:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfd880, transno 4295068959 00000004:00080000:3.0:1785220586.222954:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bef100, transno 4295068960 00000004:00080000:3.0:1785220586.222957:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfea00, transno 4295068965 00000004:00080000:3.0:1785220586.222959:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef5f80, transno 4295068966 00000004:00080000:3.0:1785220586.222961:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef4e00, transno 4295068968 00000004:00080000:3.0:1785220586.222964:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93572a00, transno 4295068970 00000004:00080000:3.0:1785220586.222966:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93572d80, transno 4295068974 00000004:00080000:3.0:1785220586.222968:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93572680, transno 4295068975 00000004:00080000:3.0:1785220586.222970:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999be300, transno 4295068977 00000004:00080000:3.0:1785220586.222973:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93571880, transno 4295068978 00000004:00080000:3.0:1785220586.222975:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986efb80, transno 4295068982 00000004:00080000:3.0:1785220586.222978:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1e300, transno 4295068984 00000004:00080000:3.0:1785220586.222981:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9641b800, transno 4295068986 00000004:00080000:3.0:1785220586.222983:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1fb80, transno 4295068987 00000004:00080000:3.0:1785220586.222986:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ef480, transno 4295068991 00000004:00080000:3.0:1785220586.222988:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1d180, transno 4295068994 00000004:00080000:3.0:1785220586.222991:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98a1c000, transno 4295068995 00000004:00080000:3.0:1785220586.222993:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d999bd180, transno 4295068996 00000004:00080000:3.0:1785220586.222996:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d63a300, transno 4295068999 00000004:00080000:3.0:1785220586.222999:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d639c00, transno 4295069002 00000004:00080000:3.0:1785220586.223002:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49ce00, transno 4295069003 00000100:00100000:2.0:1785220586.223055:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220586.223073:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d63aa00, transno 4295069004 00000004:00080000:3.0:1785220586.223077:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49f100, transno 4295069008 00000004:00080000:3.0:1785220586.223080:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49dc00, transno 4295069010 00000004:00080000:3.0:1785220586.223082:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49f800, transno 4295069011 00000100:00100000:2.0:1785220586.223149:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1871938510817152:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.223211:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1785220586.223235:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d8e5b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1871938510817664:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.223258:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1785220586.223401:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ac380, transno 4295069013 00000004:00080000:3.0:1785220586.223405:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362ad80, transno 4295069015 00000004:00080000:3.0:1785220586.223407:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49df80, transno 4295069017 00000004:00080000:3.0:1785220586.223409:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49c380, transno 4295069020 00000004:00080000:3.0:1785220586.223411:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362aa00, transno 4295069021 00000004:00080000:3.0:1785220586.223414:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93629f80, transno 4295069023 00000004:00080000:3.0:1785220586.223416:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c61f80, transno 4295069028 00000004:00080000:3.0:1785220586.223419:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa65500, transno 4295069030 00000004:00080000:3.0:1785220586.223421:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c60a80, transno 4295069031 00000004:00080000:3.0:1785220586.223424:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e30e00, transno 4295069032 00000004:00080000:3.0:1785220586.223427:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979ba680, transno 4295069033 00000004:00080000:3.0:1785220586.223429:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b46300, transno 4295069034 00000004:00080000:3.0:1785220586.223431:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c60000, transno 4295069035 00000004:00080000:3.0:1785220586.223434:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc2a00, transno 4295069036 00000004:00080000:3.0:1785220586.223436:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b46d80, transno 4295069040 00000004:00080000:3.0:1785220586.223439:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995d880, transno 4295069042 00000004:00080000:3.0:1785220586.223442:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f3480, transno 4295069045 00000004:00080000:3.0:1785220586.223445:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a8700, transno 4295069047 00000004:00080000:3.0:1785220586.223447:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de7100, transno 4295069048 00000004:00080000:3.0:1785220586.223450:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a9500, transno 4295069050 00000004:00080000:3.0:1785220586.223452:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49c700, transno 4295069051 00000004:00080000:3.0:1785220586.223455:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735ea00, transno 4295069053 00000004:00080000:3.0:1785220586.223457:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49d180, transno 4295069054 00000004:00080000:3.0:1785220586.223459:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92164000, transno 4295069059 00000004:00080000:3.0:1785220586.223462:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92166d80, transno 4295069061 00000004:00080000:3.0:1785220586.223464:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901ad180, transno 4295069062 00000004:00080000:3.0:1785220586.223467:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92164380, transno 4295069064 00000004:00080000:3.0:1785220586.223469:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92166a00, transno 4295069065 00000004:00080000:3.0:1785220586.223472:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dd180, transno 4295069069 00000004:00080000:3.0:1785220586.223474:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977ddf80, transno 4295069070 00000004:00080000:3.0:1785220586.223477:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92167800, transno 4295069072 00000004:00080000:3.0:1785220586.223479:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e87100, transno 4295069074 00000004:00080000:3.0:1785220586.223482:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca9500, transno 4295069077 00000004:00080000:3.0:1785220586.223484:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca8000, transno 4295069078 00000004:00080000:3.0:1785220586.223487:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5edc00, transno 4295069080 00000004:00080000:3.0:1785220586.223490:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985eb480, transno 4295069083 00000004:00080000:3.0:1785220586.223493:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca9880, transno 4295069084 00000004:00080000:3.0:1785220586.223495:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa67b80, transno 4295069086 00000004:00080000:3.0:1785220586.223497:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa65880, transno 4295069087 00000004:00080000:3.0:1785220586.223500:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9adb1f80, transno 4295069089 00000004:00080000:3.0:1785220586.223503:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e9c00, transno 4295069090 00000004:00080000:3.0:1785220586.223505:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9adb3480, transno 4295069094 00000004:00080000:3.0:1785220586.223508:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f54380, transno 4295069095 00000004:00080000:3.0:1785220586.223511:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7c380, transno 4295069098 00000004:00080000:3.0:1785220586.223513:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480c700, transno 4295069100 00000004:00080000:3.0:1785220586.223515:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480ea00, transno 4295069101 00000004:00080000:3.0:1785220586.223518:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7e300, transno 4295069103 00000004:00080000:3.0:1785220586.223520:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480e300, transno 4295069106 00000004:00080000:3.0:1785220586.223522:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480ed80, transno 4295069107 00000004:00080000:3.0:1785220586.223525:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adb100, transno 4295069108 00000004:00080000:3.0:1785220586.223527:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d1180, transno 4295069109 00000004:00080000:3.0:1785220586.223530:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb7b480, transno 4295069112 00000004:00080000:3.0:1785220586.223532:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb7bb80, transno 4295069113 00000004:00080000:3.0:1785220586.223534:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0de300, transno 4295069116 00000004:00080000:3.0:1785220586.223537:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0dd500, transno 4295069120 00000004:00080000:3.0:1785220586.223539:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbd880, transno 4295069122 00000004:00080000:3.0:1785220586.223542:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc3480, transno 4295069125 00000004:00080000:3.0:1785220586.223544:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908bb480, transno 4295069126 00000004:00080000:3.0:1785220586.223547:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c60380, transno 4295069127 00000004:00080000:3.0:1785220586.223549:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856d180, transno 4295069128 00000004:00080000:3.0:1785220586.223552:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856f100, transno 4295069129 00000004:00080000:3.0:1785220586.223554:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb78700, transno 4295069134 00000004:00080000:3.0:1785220586.223557:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856fb80, transno 4295069135 00000004:00080000:3.0:1785220586.223559:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908b9f80, transno 4295069137 00000004:00080000:3.0:1785220586.223561:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6df80, transno 4295069140 00000004:00080000:3.0:1785220586.223563:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ef800, transno 4295069141 00000004:00080000:3.0:1785220586.223566:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ef480, transno 4295069142 00000004:00080000:3.0:1785220586.223569:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ee680, transno 4295069143 00000004:00080000:3.0:1785220586.223571:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901adc00, transno 4295069146 00000004:00080000:3.0:1785220586.223573:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c81880, transno 4295069149 00000004:00080000:3.0:1785220586.223576:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be2680, transno 4295069150 00000004:00080000:3.0:1785220586.223578:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91506d80, transno 4295069152 00000004:00080000:3.0:1785220586.223581:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91505880, transno 4295069153 00000004:00080000:3.0:1785220586.223583:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93742680, transno 4295069154 00000004:00080000:3.0:1785220586.223585:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98624700, transno 4295069155 00000004:00080000:3.0:1785220586.223587:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98626680, transno 4295069158 00000004:00080000:3.0:1785220586.223590:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98625f80, transno 4295069160 00000004:00080000:3.0:1785220586.223593:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afb100, transno 4295069165 00000100:00100000:0.0:1785220586.223593:0:13921:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8e5b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1871938510816768:12345-0@lo:400:kworker.0 Request processed in 5408us (5715us total) trans 0 rc 0/0 00000004:00080000:3.0:1785220586.223595:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af9880, transno 4295069166 00000004:00080000:3.0:1785220586.223597:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6c380, transno 4295069167 00000004:00080000:3.0:1785220586.223601:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91504a80, transno 4295069169 00000004:00080000:3.0:1785220586.223603:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1dc00, transno 4295069171 00000100:00100000:0.0:1785220586.223605:0:13921:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1354 00000004:00080000:3.0:1785220586.223606:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6d880, transno 4295069172 00000004:00080000:3.0:1785220586.223608:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afad80, transno 4295069173 00000004:00080000:3.0:1785220586.223610:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1c700, transno 4295069174 00000004:00080000:3.0:1785220586.223612:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c83b80, transno 4295069175 00000004:00080000:3.0:1785220586.223614:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984f480, transno 4295069179 00000004:00080000:3.0:1785220586.223617:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984c700, transno 4295069182 00000004:00080000:3.0:1785220586.223619:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984dc00, transno 4295069184 00000004:00080000:3.0:1785220586.223621:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73a680, transno 4295069188 00000004:00080000:3.0:1785220586.223623:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979a6300, transno 4295069189 00000004:00080000:3.0:1785220586.223625:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b9180, transno 4295069190 00000004:00080000:3.0:1785220586.223627:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979b9500, transno 4295069191 00000004:00080000:3.0:1785220586.223629:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979a5500, transno 4295069194 00000004:00080000:3.0:1785220586.223632:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cdf80, transno 4295069195 00000100:00100000:0.0:1785220586.223632:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817152 00000004:00080000:3.0:1785220586.223634:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91173100, transno 4295069197 00000004:00080000:3.0:1785220586.223637:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853d180, transno 4295069200 00000100:00100000:0.0:1785220586.223637:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817152 00000004:00080000:3.0:1785220586.223639:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa7800, transno 4295069202 00000004:00080000:3.0:1785220586.223641:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cf480, transno 4295069204 00000004:00080000:3.0:1785220586.223643:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfeaa00, transno 4295069206 00000004:00080000:3.0:1785220586.223646:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cc380, transno 4295069208 00000004:00080000:3.0:1785220586.223648:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a7480, transno 4295069212 00000004:00080000:3.0:1785220586.223650:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa4e00, transno 4295069213 00000004:00080000:3.0:1785220586.223653:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6ad80, transno 4295069215 00000004:00080000:3.0:1785220586.223655:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991a6d80, transno 4295069216 00000004:00080000:3.0:1785220586.223657:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ed500, transno 4295069222 00000004:00080000:3.0:1785220586.223660:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95421f80, transno 4295069223 00000004:00080000:3.0:1785220586.223662:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0eed80, transno 4295069224 00000004:00080000:3.0:1785220586.223664:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f69880, transno 4295069227 00000004:00080000:3.0:1785220586.223667:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901aea00, transno 4295069228 00000004:00080000:3.0:1785220586.223669:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90407480, transno 4295069229 00000004:00080000:3.0:1785220586.223671:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f56300, transno 4295069230 00000004:00080000:3.0:1785220586.223674:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ee680, transno 4295069231 00000004:00080000:3.0:1785220586.223676:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa9880, transno 4295069236 00000004:00080000:3.0:1785220586.223679:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa9500, transno 4295069237 00000100:00100000:0.0:1785220586.223691:0:13921:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817664 00000004:00080000:3.0:1785220586.223692:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91507800, transno 4295069240 00000004:00080000:3.0:1785220586.223694:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de4000, transno 4295069243 00000100:00100000:0.0:1785220586.223694:0:13921:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817664 00000004:00080000:3.0:1785220586.223697:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90404a80, transno 4295069244 00000004:00080000:3.0:1785220586.223700:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af5c00, transno 4295069245 00000004:00080000:3.0:1785220586.223702:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f739180, transno 4295069248 00000004:00080000:3.0:1785220586.223705:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa7100, transno 4295069249 00000004:00080000:3.0:1785220586.223707:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73ad80, transno 4295069250 00000004:00080000:3.0:1785220586.223710:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480fb80, transno 4295069251 00000004:00080000:3.0:1785220586.223712:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faaa680, transno 4295069252 00000004:00080000:3.0:1785220586.223714:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985ead80, transno 4295069255 00000100:00100000:0.0:1785220586.223715:0:13921:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1358 00000004:00080000:3.0:1785220586.223717:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e9500, transno 4295069256 00000004:00080000:3.0:1785220586.223720:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93632d80, transno 4295069258 00000100:00100000:0.0:1785220586.223722:0:13921:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d96798380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1871938510817152:12345-0@lo:400:kworker.0 00000004:00080000:3.0:1785220586.223723:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2aa00, transno 4295069264 00000004:00080000:3.0:1785220586.223725:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b28e00, transno 4295069265 00000004:00080000:3.0:1785220586.223727:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2b800, transno 4295069267 00000004:00080000:3.0:1785220586.223729:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364b480, transno 4295069268 00000004:00080000:3.0:1785220586.223732:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b2ad80, transno 4295069269 00000004:00080000:3.0:1785220586.223733:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93648700, transno 4295069270 00000004:00080000:3.0:1785220586.223736:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364a300, transno 4295069273 00000004:00080000:3.0:1785220586.223738:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87d44380, transno 4295069277 00000004:00080000:3.0:1785220586.223740:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcab800, transno 4295069279 00000004:00080000:3.0:1785220586.223743:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af4000, transno 4295069281 00000004:00080000:3.0:1785220586.223745:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad1500, transno 4295069283 00000004:00080000:3.0:1785220586.223748:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480f480, transno 4295069284 00000004:00080000:3.0:1785220586.223750:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986fb80, transno 4295069286 00000100:00100000:0.0:1785220586.223763:0:13921:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d96798380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1871938510817152:12345-0@lo:400:kworker.0 Request processed in 41us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220586.223769:0:13921:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1358 00000100:00100000:0.0:1785220586.223779:0:13921:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1359 00000100:00100000:0.0:1785220586.223785:0:13921:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9679ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1871938510817664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1785220586.223811:0:13921:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9679ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_026:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1871938510817664:12345-0@lo:400:kworker.0 Request processed in 26us (552us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220586.223817:0:13921:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1359 00000100:00080000:0.0:1785220586.223844:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1785220586.223849:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1785220586.223906:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510817024 00000100:00100000:0.0:1785220586.223910:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510817024 00000100:00100000:0.0:1785220586.223936:0:13815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12576 00000100:00100000:0.0:1785220586.223941:0:13815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1871938510817024:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1785220586.223978:0:13815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1871938510817024:12345-0@lo:400:kworker.0 Request processed in 37us (6044us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220586.223984:0:13815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12576 00000100:00100000:0.0:1785220586.224014:0:3652:0:(client.c:3112:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4295069983 gen 2 00000004:00080000:0.0:1785220586.224018:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa8700, transno 4295068791 00000004:00080000:0.0:1785220586.224026:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9544ad80, transno 4295068792 00000004:00080000:0.0:1785220586.224029:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfb480, transno 4295068793 00000004:00080000:0.0:1785220586.224031:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96769c00, transno 4295068796 00000004:00080000:0.0:1785220586.224034:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96768000, transno 4295068797 00000004:00080000:0.0:1785220586.224036:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfb100, transno 4295068798 00000004:00080000:0.0:1785220586.224038:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb1500, transno 4295068801 00000004:00080000:0.0:1785220586.224041:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7f480, transno 4295068802 00000004:00080000:0.0:1785220586.224043:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbe300, transno 4295068804 00000004:00080000:0.0:1785220586.224046:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91582300, transno 4295068808 00000004:00080000:0.0:1785220586.224048:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91580000, transno 4295068810 00000004:00080000:0.0:1785220586.224050:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1df80, transno 4295068812 00000004:00080000:0.0:1785220586.224053:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfb800, transno 4295068814 00000004:00080000:0.0:1785220586.224055:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91581880, transno 4295068815 00000004:00080000:0.0:1785220586.224057:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf8000, transno 4295068817 00000004:00080000:0.0:1785220586.224060:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1f480, transno 4295068818 00000004:00080000:0.0:1785220586.224062:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb1180, transno 4295068821 00000004:00080000:0.0:1785220586.224065:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c973800, transno 4295068828 00000004:00080000:0.0:1785220586.224067:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676b800, transno 4295068830 00000004:00080000:0.0:1785220586.224070:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c973100, transno 4295068832 00000004:00080000:0.0:1785220586.224072:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676a680, transno 4295068833 00000004:00080000:0.0:1785220586.224075:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c970a80, transno 4295068837 00000004:00080000:0.0:1785220586.224077:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96769880, transno 4295068839 00000004:00080000:0.0:1785220586.224079:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96768380, transno 4295068840 00000004:00080000:0.0:1785220586.224081:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96769180, transno 4295068841 00000004:00080000:0.0:1785220586.224109:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95451c00, transno 4295068844 00000004:00080000:0.0:1785220586.224114:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c168380, transno 4295068847 00000004:00080000:0.0:1785220586.224116:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95450a80, transno 4295068849 00000004:00080000:0.0:1785220586.224119:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95450700, transno 4295068851 00000004:00080000:0.0:1785220586.224121:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be1180, transno 4295068854 00000004:00080000:0.0:1785220586.224124:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce8e00, transno 4295068855 00000004:00080000:0.0:1785220586.224126:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be3480, transno 4295068857 00000004:00080000:0.0:1785220586.224128:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be0000, transno 4295068859 00000004:00080000:0.0:1785220586.224131:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95452300, transno 4295068862 00000004:00080000:0.0:1785220586.224133:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbdc00, transno 4295068864 00000004:00080000:0.0:1785220586.224136:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364a680, transno 4295068867 00000004:00080000:0.0:1785220586.224139:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b46680, transno 4295068868 00000004:00080000:0.0:1785220586.224141:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa7b80, transno 4295068870 00000004:00080000:0.0:1785220586.224144:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bec700, transno 4295068872 00000004:00080000:0.0:1785220586.224146:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98410700, transno 4295068873 00000004:00080000:0.0:1785220586.224148:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cfa300, transno 4295068875 00000004:00080000:0.0:1785220586.224151:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1f100, transno 4295068876 00000004:00080000:0.0:1785220586.224153:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90405880, transno 4295068879 00000004:00080000:0.0:1785220586.224156:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93628e00, transno 4295068882 00000004:00080000:0.0:1785220586.224159:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90405c00, transno 4295068883 00000004:00080000:0.0:1785220586.224162:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce8380, transno 4295068887 00000004:00080000:0.0:1785220586.224165:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e2a00, transno 4295068889 00000004:00080000:0.0:1785220586.224167:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ece00, transno 4295068890 00000004:00080000:0.0:1785220586.224170:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90404380, transno 4295068891 00000004:00080000:0.0:1785220586.224172:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95423800, transno 4295068892 00000004:00080000:0.0:1785220586.224175:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cea680, transno 4295068895 00000004:00080000:0.0:1785220586.224178:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91580380, transno 4295068896 00000004:00080000:0.0:1785220586.224180:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91580a80, transno 4295068899 00000004:00080000:0.0:1785220586.224182:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95420380, transno 4295068900 00000004:00080000:0.0:1785220586.224185:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bd500, transno 4295068901 00000004:00080000:0.0:1785220586.224188:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ce9c00, transno 4295068902 00000004:00080000:0.0:1785220586.224190:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9364bb80, transno 4295068905 00000004:00080000:0.0:1785220586.224193:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d793800, transno 4295068909 00000004:00080000:0.0:1785220586.224196:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d791180, transno 4295068911 00000004:00080000:0.0:1785220586.224198:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f69f80, transno 4295068912 00000004:00080000:0.0:1785220586.224200:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc1500, transno 4295068915 00000004:00080000:0.0:1785220586.224203:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d790a80, transno 4295068916 00000004:00080000:0.0:1785220586.224205:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f6aa00, transno 4295068917 00000004:00080000:0.0:1785220586.224207:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f68000, transno 4295068918 00000004:00080000:0.0:1785220586.224209:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bfc2680, transno 4295068919 00000004:00080000:0.0:1785220586.224211:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49f480, transno 4295068920 00000004:00080000:0.0:1785220586.224214:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49ed80, transno 4295068923 00000004:00080000:0.0:1785220586.224217:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986f480, transno 4295068924 00000004:00080000:0.0:1785220586.224219:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901af800, transno 4295068925 00000004:00080000:0.0:1785220586.224221:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91505180, transno 4295068930 00000004:00080000:0.0:1785220586.224224:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bf480, transno 4295068931 00000004:00080000:0.0:1785220586.224227:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49e300, transno 4295068932 00000004:00080000:0.0:1785220586.224229:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91507b80, transno 4295068934 00000004:00080000:0.0:1785220586.224232:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986df80, transno 4295068937 00000004:00080000:0.0:1785220586.224234:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98f69c00, transno 4295068938 00000004:00080000:0.0:1785220586.224237:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcaa680, transno 4295068939 00000004:00080000:0.0:1785220586.224239:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca9180, transno 4295068940 00000004:00080000:0.0:1785220586.224241:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bdf80, transno 4295068945 00000004:00080000:0.0:1785220586.224244:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bce00, transno 4295068946 00000004:00080000:0.0:1785220586.224246:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bc700, transno 4295068950 00000004:00080000:0.0:1785220586.224249:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936be300, transno 4295068952 00000004:00080000:0.0:1785220586.224251:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e8380, transno 4295068953 00000004:00080000:0.0:1785220586.224253:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0ab100, transno 4295068954 00000004:00080000:0.0:1785220586.224255:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0ab800, transno 4295068956 00000004:00080000:0.0:1785220586.224257:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97588a80, transno 4295068961 00000004:00080000:0.0:1785220586.224260:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a9c00, transno 4295068962 00000004:00080000:0.0:1785220586.224262:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e8000, transno 4295068963 00000004:00080000:0.0:1785220586.224264:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9758bb80, transno 4295068964 00000004:00080000:0.0:1785220586.224267:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d5500, transno 4295068966 00000004:00080000:0.0:1785220586.224269:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d4000, transno 4295068967 00000004:00080000:0.0:1785220586.224272:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979ea300, transno 4295068968 00000004:00080000:0.0:1785220586.224274:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979eb480, transno 4295068973 00000004:00080000:0.0:1785220586.224277:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d7b80, transno 4295068974 00000004:00080000:3.0:1785220586.224404:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480d880, transno 4295069287 00000004:00080000:3.0:1785220586.224408:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93633100, transno 4295069289 00000004:00080000:3.0:1785220586.224410:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93633800, transno 4295069291 00000004:00080000:3.0:1785220586.224412:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985eaa00, transno 4295069295 00000004:00080000:3.0:1785220586.224414:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfea680, transno 4295069298 00000004:00080000:3.0:1785220586.224417:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afaa00, transno 4295069299 00000004:00080000:3.0:1785220586.224419:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfeb480, transno 4295069300 00000004:00080000:3.0:1785220586.224422:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c7d180, transno 4295069301 00000004:00080000:3.0:1785220586.224425:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99024000, transno 4295069302 00000004:00080000:3.0:1785220586.224427:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91506680, transno 4295069303 00000004:00080000:3.0:1785220586.224430:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad3b80, transno 4295069306 00000004:00080000:3.0:1785220586.224432:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad1f80, transno 4295069310 00000004:00080000:3.0:1785220586.224435:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d2d80, transno 4295069313 00000004:00080000:3.0:1785220586.224438:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d2300, transno 4295069315 00000004:00080000:3.0:1785220586.224440:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb34380, transno 4295069317 00000004:00080000:3.0:1785220586.224443:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb35180, transno 4295069318 00000004:00080000:3.0:1785220586.224446:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93648e00, transno 4295069324 00000004:00080000:3.0:1785220586.224449:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93649500, transno 4295069326 00000004:00080000:3.0:1785220586.224451:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb37100, transno 4295069327 00000004:00080000:3.0:1785220586.224453:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb37800, transno 4295069328 00000004:00080000:3.0:1785220586.224456:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb34e00, transno 4295069329 00000004:00080000:3.0:1785220586.224458:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95449500, transno 4295069330 00000004:00080000:3.0:1785220586.224460:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb35c00, transno 4295069333 00000004:00080000:3.0:1785220586.224462:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbfb80, transno 4295069336 00000004:00080000:3.0:1785220586.224466:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00d880, transno 4295069337 00000004:00080000:3.0:1785220586.224468:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00ea00, transno 4295069340 00000004:00080000:3.0:1785220586.224471:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa59c00, transno 4295069342 00000004:00080000:3.0:1785220586.224474:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86310a80, transno 4295069343 00000004:00080000:3.0:1785220586.224476:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86311f80, transno 4295069345 00000004:00080000:3.0:1785220586.224479:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86310380, transno 4295069348 00000004:00080000:3.0:1785220586.224481:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86312a00, transno 4295069349 00000004:00080000:3.0:1785220586.224483:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86313800, transno 4295069350 00000004:00080000:3.0:1785220586.224485:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb34000, transno 4295069353 00000004:00080000:3.0:1785220586.224487:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb36680, transno 4295069356 00000004:00080000:3.0:1785220586.224489:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95471f80, transno 4295069357 00000004:00080000:3.0:1785220586.224492:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e773100, transno 4295069360 00000004:00080000:3.0:1785220586.224494:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e771c00, transno 4295069361 00000004:00080000:3.0:1785220586.224497:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c929500, transno 4295069364 00000004:00080000:3.0:1785220586.224499:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c929c00, transno 4295069367 00000100:00100000:3.0:1785220586.224515:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9a707480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1871938510818048:0@lo:13:osp-pre-1-1.0 00000004:00080000:0.0:1785220586.225970:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d4e00, transno 4295068978 00000004:00080000:0.0:1785220586.225974:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97957100, transno 4295068979 00000004:00080000:0.0:1785220586.225976:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d7800, transno 4295068980 00000004:00080000:0.0:1785220586.225979:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d5880, transno 4295068982 00000004:00080000:0.0:1785220586.225982:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d5180, transno 4295068984 00000004:00080000:0.0:1785220586.225984:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92369500, transno 4295068989 00000004:00080000:0.0:1785220586.225987:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995ea00, transno 4295068990 00000004:00080000:0.0:1785220586.225989:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995c700, transno 4295068991 00000004:00080000:0.0:1785220586.225992:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0aaa00, transno 4295068994 00000004:00080000:0.0:1785220586.225994:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0aa680, transno 4295068995 00000004:00080000:0.0:1785220586.225996:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995c000, transno 4295068996 00000004:00080000:0.0:1785220586.225999:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684ed80, transno 4295068997 00000004:00080000:0.0:1785220586.226001:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684ce00, transno 4295068999 00000004:00080000:0.0:1785220586.226004:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684f800, transno 4295069001 00000004:00080000:0.0:1785220586.226006:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684f100, transno 4295069003 00000004:00080000:0.0:1785220586.226008:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684d500, transno 4295069004 00000004:00080000:0.0:1785220586.226011:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0abb80, transno 4295069005 00000004:00080000:0.0:1785220586.226013:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684ca80, transno 4295069006 00000004:00080000:0.0:1785220586.226015:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6ed80, transno 4295069011 00000004:00080000:0.0:1785220586.226018:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6c700, transno 4295069012 00000004:00080000:0.0:1785220586.226020:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a8000, transno 4295069015 00000004:00080000:0.0:1785220586.226022:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a8a80, transno 4295069016 00000004:00080000:0.0:1785220586.226025:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6f480, transno 4295069019 00000004:00080000:0.0:1785220586.226028:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6c000, transno 4295069020 00000004:00080000:0.0:1785220586.226030:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6d500, transno 4295069023 00000004:00080000:0.0:1785220586.226034:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6ea00, transno 4295069025 00000004:00080000:0.0:1785220586.226036:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00ce00, transno 4295069027 00000004:00080000:0.0:1785220586.226038:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6df80, transno 4295069028 00000004:00080000:0.0:1785220586.226041:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6f800, transno 4295069030 00000004:00080000:0.0:1785220586.226044:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93c63800, transno 4295069031 00000004:00080000:0.0:1785220586.226046:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9236a680, transno 4295069036 00000004:00080000:0.0:1785220586.226048:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d6680, transno 4295069038 00000004:00080000:0.0:1785220586.226051:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d4380, transno 4295069040 00000004:00080000:0.0:1785220586.226053:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d790000, transno 4295069042 00000004:00080000:0.0:1785220586.226055:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d792d80, transno 4295069043 00000004:00080000:0.0:1785220586.226058:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d6a00, transno 4295069044 00000004:00080000:0.0:1785220586.226061:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d790380, transno 4295069047 00000004:00080000:0.0:1785220586.226063:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d791500, transno 4295069048 00000004:00080000:0.0:1785220586.226065:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97440000, transno 4295069052 00000004:00080000:0.0:1785220586.226068:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97442680, transno 4295069053 00000004:00080000:0.0:1785220586.226070:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97440700, transno 4295069055 00000004:00080000:0.0:1785220586.226072:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97441c00, transno 4295069057 00000004:00080000:0.0:1785220586.226074:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bf100, transno 4295069059 00000004:00080000:0.0:1785220586.226077:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977df100, transno 4295069060 00000004:00080000:0.0:1785220586.226079:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97443100, transno 4295069062 00000004:00080000:0.0:1785220586.226082:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dc380, transno 4295069063 00000004:00080000:0.0:1785220586.226101:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bd500, transno 4295069064 00000004:00080000:0.0:1785220586.226104:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8e300, transno 4295069066 00000004:00080000:0.0:1785220586.226107:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d936bed80, transno 4295069068 00000004:00080000:0.0:1785220586.226109:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f1c00, transno 4295069070 00000004:00080000:0.0:1785220586.226112:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856f800, transno 4295069075 00000004:00080000:0.0:1785220586.226115:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735f100, transno 4295069076 00000004:00080000:0.0:1785220586.226117:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735ed80, transno 4295069077 00000004:00080000:0.0:1785220586.226120:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979e8e00, transno 4295069078 00000004:00080000:0.0:1785220586.226123:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977aad80, transno 4295069080 00000004:00080000:0.0:1785220586.226125:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735c700, transno 4295069083 00000004:00080000:0.0:1785220586.226128:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977a8e00, transno 4295069086 00000004:00080000:0.0:1785220586.226131:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f54a80, transno 4295069089 00000004:00080000:0.0:1785220586.226133:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c2b800, transno 4295069091 00000004:00080000:0.0:1785220586.226136:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856f480, transno 4295069092 00000004:00080000:0.0:1785220586.226138:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f56d80, transno 4295069093 00000004:00080000:0.0:1785220586.226140:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908baa00, transno 4295069094 00000004:00080000:0.0:1785220586.226143:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e5b4e00, transno 4295069095 00000004:00080000:0.0:1785220586.226146:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f54700, transno 4295069096 00000004:00080000:0.0:1785220586.226148:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92166680, transno 4295069098 00000004:00080000:0.0:1785220586.226151:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97443480, transno 4295069100 00000004:00080000:0.0:1785220586.226153:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbf100, transno 4295069101 00000004:00080000:0.0:1785220586.226156:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908bb800, transno 4295069103 00000004:00080000:0.0:1785220586.226158:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e5b4a80, transno 4295069109 00000004:00080000:0.0:1785220586.226160:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9995e680, transno 4295069110 00000004:00080000:0.0:1785220586.226163:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684d180, transno 4295069111 00000004:00080000:0.0:1785220586.226165:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684df80, transno 4295069112 00000004:00080000:0.0:1785220586.226167:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684c700, transno 4295069114 00000004:00080000:0.0:1785220586.226169:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c29500, transno 4295069118 00000004:00080000:0.0:1785220586.226172:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92167480, transno 4295069119 00000004:00080000:0.0:1785220586.226174:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbf800, transno 4295069121 00000004:00080000:0.0:1785220586.226177:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de4a80, transno 4295069122 00000004:00080000:0.0:1785220586.226179:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794c700, transno 4295069126 00000004:00080000:0.0:1785220586.226181:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960eb735d880, transno 4295069127 00000004:00080000:0.0:1785220586.226184:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794d880, transno 4295069131 00000004:00080000:0.0:1785220586.226186:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794e300, transno 4295069132 00000004:00080000:0.0:1785220586.226189:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6e680, transno 4295069135 00000004:00080000:0.0:1785220586.226191:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794d500, transno 4295069136 00000004:00080000:0.0:1785220586.226194:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6ce00, transno 4295069137 00000004:00080000:0.0:1785220586.226197:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6ed80, transno 4295069138 00000004:00080000:0.0:1785220586.226199:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6ca80, transno 4295069141 00000004:00080000:0.0:1785220586.226202:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684fb80, transno 4295069143 00000004:00080000:0.0:1785220586.226204:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d0700, transno 4295069149 00000004:00080000:0.0:1785220586.226207:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d2a00, transno 4295069150 00000004:00080000:0.0:1785220586.226209:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d3800, transno 4295069151 00000004:00080000:0.0:1785220586.226211:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d1880, transno 4295069152 00000004:00080000:0.0:1785220586.226214:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d0000, transno 4295069153 00000004:00080000:0.0:1785220586.226216:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794df80, transno 4295069158 00000004:00080000:0.0:1785220586.226219:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d2300, transno 4295069161 00000004:00080000:0.0:1785220586.226221:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d0380, transno 4295069163 00000004:00080000:0.0:1785220586.226224:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c548000, transno 4295069165 00000004:00080000:0.0:1785220586.226227:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c548700, transno 4295069167 00000004:00080000:0.0:1785220586.226229:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c549880, transno 4295069168 00000004:00080000:0.0:1785220586.226231:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c54aa00, transno 4295069169 00000004:00080000:0.0:1785220586.226234:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c548380, transno 4295069171 00000004:00080000:0.0:1785220586.226236:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c549c00, transno 4295069175 00000004:00080000:0.0:1785220586.226238:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc0000, transno 4295069178 00000004:00080000:0.0:1785220586.226240:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ec000, transno 4295069179 00000004:00080000:0.0:1785220586.226243:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e84380, transno 4295069182 00000004:00080000:0.0:1785220586.226245:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e85180, transno 4295069184 00000004:00080000:0.0:1785220586.226247:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96418380, transno 4295069185 00000004:00080000:0.0:1785220586.226250:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e86a00, transno 4295069187 00000004:00080000:0.0:1785220586.226252:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e86300, transno 4295069191 00000004:00080000:0.0:1785220586.226254:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc3480, transno 4295069192 00000004:00080000:0.0:1785220586.226256:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ec700, transno 4295069197 00000004:00080000:0.0:1785220586.226258:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc3b80, transno 4295069198 00000004:00080000:0.0:1785220586.226261:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986eea00, transno 4295069199 00000004:00080000:0.0:1785220586.226263:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5ec380, transno 4295069202 00000004:00080000:0.0:1785220586.226266:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92368380, transno 4295069203 00000004:00080000:0.0:1785220586.226268:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ef800, transno 4295069206 00000004:00080000:0.0:1785220586.226270:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b34bb80, transno 4295069209 00000004:00080000:0.0:1785220586.226272:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fd500, transno 4295069214 00000004:00080000:0.0:1785220586.226275:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fd880, transno 4295069215 00000004:00080000:0.0:1785220586.226277:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fdc00, transno 4295069217 00000100:00100000:1.0:1785220586.226459:0:6519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510816640 00000100:00100000:1.0:1785220586.226467:0:6519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510816640 00000100:00100000:1.0:1785220586.226528:0:6519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:1.0:1785220586.226538:0:6519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac52b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4add7b40-ab9d-45be-a6c9-eb07929d24cd+24:3655:x1871938510816640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1785220586.226608:0:6519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac52b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4add7b40-ab9d-45be-a6c9-eb07929d24cd+24:3655:x1871938510816640:12345-0@lo:400:kworker.0 Request processed in 71us (3555us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220586.226615:0:6519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:1.0:1785220586.226691:0:12310:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d96799f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1871938510817280:12345-0@lo:400:kworker.0 Request processed in 8427us (8757us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220586.226700:0:12310:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1355 00000004:00080000:0.0:1785220586.227380:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fe680, transno 4295069218 00000004:00080000:0.0:1785220586.227383:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fdf80, transno 4295069220 00000004:00080000:0.0:1785220586.227386:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e73b80, transno 4295069222 00000004:00080000:0.0:1785220586.227388:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efb100, transno 4295069224 00000004:00080000:0.0:1785220586.227391:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ed500, transno 4295069225 00000004:00080000:0.0:1785220586.227394:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ad8a80, transno 4295069228 00000004:00080000:0.0:1785220586.227396:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ad9880, transno 4295069229 00000004:00080000:0.0:1785220586.227398:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ada300, transno 4295069231 00000004:00080000:0.0:1785220586.227401:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ad9c00, transno 4295069234 00000004:00080000:0.0:1785220586.227403:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98adad80, transno 4295069236 00000004:00080000:0.0:1785220586.227406:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ada680, transno 4295069239 00000004:00080000:0.0:1785220586.227408:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e70700, transno 4295069240 00000004:00080000:0.0:1785220586.227411:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98ad8380, transno 4295069242 00000004:00080000:0.0:1785220586.227413:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b98700, transno 4295069244 00000004:00080000:0.0:1785220586.227416:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794c380, transno 4295069248 00000004:00080000:0.0:1785220586.227418:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56ce00, transno 4295069249 00000004:00080000:0.0:1785220586.227420:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b9a680, transno 4295069250 00000004:00080000:0.0:1785220586.227422:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fc000, transno 4295069253 00000004:00080000:0.0:1785220586.227424:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56fb80, transno 4295069254 00000004:00080000:0.0:1785220586.227426:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94ef3b80, transno 4295069256 00000004:00080000:0.0:1785220586.227429:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99024700, transno 4295069258 00000004:00080000:0.0:1785220586.227431:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efb800, transno 4295069261 00000004:00080000:0.0:1785220586.227433:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87efaa00, transno 4295069262 00000004:00080000:0.0:1785220586.227436:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875dc00, transno 4295069265 00000004:00080000:0.0:1785220586.227438:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875c000, transno 4295069266 00000004:00080000:0.0:1785220586.227441:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875d180, transno 4295069268 00000004:00080000:0.0:1785220586.227443:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875e680, transno 4295069269 00000004:00080000:0.0:1785220586.227446:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875df80, transno 4295069270 00000004:00080000:0.0:1785220586.227448:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875ca80, transno 4295069273 00000004:00080000:0.0:1785220586.227451:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875f100, transno 4295069274 00000004:00080000:0.0:1785220586.227466:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fd180, transno 4295069276 00000004:00080000:0.0:1785220586.227470:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ea684c380, transno 4295069277 00000004:00080000:0.0:1785220586.227472:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964ddc00, transno 4295069281 00000004:00080000:0.0:1785220586.227474:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875fb80, transno 4295069283 00000004:00080000:0.0:1785220586.227477:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906ab800, transno 4295069284 00000004:00080000:0.0:1785220586.227479:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a8700, transno 4295069285 00000004:00080000:0.0:1785220586.227481:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99935880, transno 4295069286 00000004:00080000:0.0:1785220586.227483:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906abb80, transno 4295069287 00000004:00080000:0.0:1785220586.227486:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d964de300, transno 4295069293 00000004:00080000:0.0:1785220586.227488:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906aa300, transno 4295069295 00000004:00080000:0.0:1785220586.227491:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875d500, transno 4295069298 00000004:00080000:0.0:1785220586.227493:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875ea00, transno 4295069299 00000004:00080000:0.0:1785220586.227495:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875f480, transno 4295069300 00000004:00080000:0.0:1785220586.227497:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875f800, transno 4295069301 00000004:00080000:0.0:1785220586.227500:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875d880, transno 4295069303 00000004:00080000:0.0:1785220586.227502:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d986ec000, transno 4295069305 00000004:00080000:0.0:1785220586.227504:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9875ce00, transno 4295069307 00000004:00080000:0.0:1785220586.227507:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8853d500, transno 4295069310 00000004:00080000:0.0:1785220586.227510:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e1500, transno 4295069311 00000004:00080000:0.0:1785220586.227512:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a9500, transno 4295069313 00000004:00080000:0.0:1785220586.227515:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e1180, transno 4295069314 00000004:00080000:0.0:1785220586.227517:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9794dc00, transno 4295069316 00000004:00080000:0.0:1785220586.227520:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4e2a00, transno 4295069320 00000004:00080000:0.0:1785220586.227522:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d1880, transno 4295069324 00000004:00080000:0.0:1785220586.227525:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d3100, transno 4295069325 00000004:00080000:0.0:1785220586.227528:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a9880, transno 4295069328 00000004:00080000:0.0:1785220586.227531:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906aad80, transno 4295069329 00000004:00080000:0.0:1785220586.227533:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d3480, transno 4295069330 00000004:00080000:0.0:1785220586.227536:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b56d880, transno 4295069333 00000004:00080000:0.0:1785220586.227538:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0dea00, transno 4295069335 00000004:00080000:0.0:1785220586.227541:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebf9d2a00, transno 4295069336 00000004:00080000:0.0:1785220586.227543:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb79880, transno 4295069338 00000004:00080000:0.0:1785220586.227546:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480df80, transno 4295069342 00000004:00080000:0.0:1785220586.227548:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d906a9180, transno 4295069343 00000004:00080000:0.0:1785220586.227551:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99027100, transno 4295069344 00000004:00080000:0.0:1785220586.227553:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc1c00, transno 4295069346 00000004:00080000:0.0:1785220586.227556:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c573480, transno 4295069349 00000004:00080000:0.0:1785220586.227558:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a4d3b80, transno 4295069351 00000004:00080000:0.0:1785220586.227560:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c571500, transno 4295069353 00000004:00080000:0.0:1785220586.227563:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c573800, transno 4295069355 00000004:00080000:0.0:1785220586.227565:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914ea00, transno 4295069356 00000004:00080000:0.0:1785220586.227568:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91171c00, transno 4295069358 00000004:00080000:0.0:1785220586.227570:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0edf80, transno 4295069360 00000004:00080000:0.0:1785220586.227573:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91173480, transno 4295069364 00000004:00080000:0.0:1785220586.227575:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91171f80, transno 4295069366 00000004:00080000:0.0:1785220586.227577:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fef7b80, transno 4295069367 00000004:00080000:0.0:1785220586.227579:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c80e00, transno 4295069368 00000004:00080000:0.0:1785220586.227582:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faab800, transno 4295069371 00000004:00080000:0.0:1785220586.227584:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91172300, transno 4295069373 00000004:00080000:0.0:1785220586.227587:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc37480, transno 4295069374 00000004:00080000:0.0:1785220586.227588:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87d47b80, transno 4295069375 00000004:00080000:0.0:1785220586.227590:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8dc35f80, transno 4295069377 00000004:00080000:0.0:1785220586.227592:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb86a00, transno 4295069382 00000004:00080000:0.0:1785220586.227595:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb85500, transno 4295069383 00000004:00080000:0.0:1785220586.227597:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90404e00, transno 4295069384 00000004:00080000:0.0:1785220586.227600:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95539880, transno 4295069387 00000004:00080000:0.0:1785220586.227602:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95538a80, transno 4295069388 00000004:00080000:0.0:1785220586.227605:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553bb80, transno 4295069390 00000004:00080000:0.0:1785220586.227607:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553b800, transno 4295069391 00000004:00080000:0.0:1785220586.227610:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af7800, transno 4295069392 00000004:00080000:0.0:1785220586.227612:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be0700, transno 4295069395 00000004:00080000:0.0:1785220586.227614:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be2300, transno 4295069399 00000004:00080000:0.0:1785220586.227616:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf8700, transno 4295069400 00000004:00080000:0.0:1785220586.227618:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf8e00, transno 4295069402 00000004:00080000:0.0:1785220586.227620:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553b480, transno 4295069403 00000004:00080000:0.0:1785220586.227622:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e2300, transno 4295069406 00000004:00080000:0.0:1785220586.227624:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c168e00, transno 4295069408 00000004:00080000:0.0:1785220586.227626:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c16b800, transno 4295069409 00000004:00080000:0.0:1785220586.227629:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb0380, transno 4295069413 00000004:00080000:0.0:1785220586.227631:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb3480, transno 4295069415 00000004:00080000:0.0:1785220586.227633:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1b480, transno 4295069418 00000004:00080000:0.0:1785220586.227635:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf9f80, transno 4295069419 00000004:00080000:0.0:1785220586.227637:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1a300, transno 4295069420 00000004:00080000:0.0:1785220586.227640:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c1ad80, transno 4295069421 00000004:00080000:0.0:1785220586.227643:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f738380, transno 4295069424 00000004:00080000:0.0:1785220586.227645:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f739500, transno 4295069426 00000004:00080000:0.0:1785220586.227647:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f73b100, transno 4295069427 00000004:00080000:0.0:1785220586.227649:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f738a80, transno 4295069429 00000004:00080000:0.0:1785220586.227651:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b6a00, transno 4295069431 00000004:00080000:0.0:1785220586.227653:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b4e00, transno 4295069433 00000004:00080000:0.0:1785220586.227656:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d904b7b80, transno 4295069434 00000004:00080000:0.0:1785220586.227658:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afbb80, transno 4295069436 00000004:00080000:0.0:1785220586.227660:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e0000, transno 4295069439 00000004:00080000:0.0:1785220586.227662:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f739880, transno 4295069441 00000004:00080000:0.0:1785220586.227664:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8b5e1500, transno 4295069442 00000004:00080000:0.0:1785220586.227667:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb1f80, transno 4295069445 00000004:00080000:0.0:1785220586.227669:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98627b80, transno 4295069447 00000004:00080000:0.0:1785220586.227672:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c19880, transno 4295069448 00000004:00080000:0.0:1785220586.227674:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c19f80, transno 4295069449 00000004:00080000:0.0:1785220586.227676:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98c19180, transno 4295069452 00000004:00080000:0.0:1785220586.227678:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98626d80, transno 4295069454 00000004:00080000:0.0:1785220586.227691:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98627100, transno 4295069456 00000004:00080000:0.0:1785220586.227694:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb3b80, transno 4295069460 00000004:00080000:0.0:1785220586.227696:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91504380, transno 4295069462 00000004:00080000:0.0:1785220586.227699:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8ceb2680, transno 4295069463 00000004:00080000:0.0:1785220586.227701:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362a680, transno 4295069464 00000004:00080000:0.0:1785220586.227704:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97b39c00, transno 4295069466 00000004:00080000:0.0:1785220586.227706:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362bb80, transno 4295069468 00000004:00080000:0.0:1785220586.227708:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93649c00, transno 4295069470 00000004:00080000:0.0:1785220586.227710:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98627800, transno 4295069472 00000004:00080000:0.0:1785220586.227711:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93628000, transno 4295069476 00000004:00080000:0.0:1785220586.227713:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95453100, transno 4295069477 00000004:00080000:0.0:1785220586.227716:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95453480, transno 4295069478 00000004:00080000:0.0:1785220586.227718:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1ce00, transno 4295069479 00000004:00080000:0.0:1785220586.227720:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfd500, transno 4295069480 00000004:00080000:0.0:1785220586.227723:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99b1c000, transno 4295069481 00000004:00080000:0.0:1785220586.227725:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfc380, transno 4295069485 00000004:00080000:0.0:1785220586.227727:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98626300, transno 4295069487 00000004:00080000:0.0:1785220586.227729:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98487800, transno 4295069489 00000004:00080000:0.0:1785220586.227732:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98484a80, transno 4295069492 00000004:00080000:0.0:1785220586.227734:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98486a00, transno 4295069493 00000004:00080000:0.0:1785220586.227736:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98486680, transno 4295069495 00000004:00080000:0.0:1785220586.227739:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98485180, transno 4295069496 00000004:00080000:0.0:1785220586.227742:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbf100, transno 4295069498 00000004:00080000:0.0:1785220586.227744:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bec380, transno 4295069502 00000004:00080000:0.0:1785220586.227747:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b15f80, transno 4295069504 00000004:00080000:0.0:1785220586.227749:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa6300, transno 4295069505 00000004:00080000:0.0:1785220586.227752:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa6680, transno 4295069510 00000004:00080000:0.0:1785220586.227754:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa4a80, transno 4295069512 00000004:00080000:0.0:1785220586.227757:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a2d2300, transno 4295069514 00000004:00080000:0.0:1785220586.227759:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa7480, transno 4295069515 00000004:00080000:0.0:1785220586.227761:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa5180, transno 4295069516 00000004:00080000:0.0:1785220586.227763:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa5f80, transno 4295069517 00000004:00080000:0.0:1785220586.227765:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a2d1c00, transno 4295069519 00000004:00080000:0.0:1785220586.227768:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a2d1f80, transno 4295069520 00000004:00080000:0.0:1785220586.227771:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b15180, transno 4295069521 00000004:00080000:0.0:1785220586.227773:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b14a80, transno 4295069523 00000004:00080000:0.0:1785220586.227775:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a2d3480, transno 4295069525 00000004:00080000:0.0:1785220586.227777:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dff800, transno 4295069527 00000004:00080000:0.0:1785220586.227779:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b15500, transno 4295069529 00000004:00080000:0.0:1785220586.227781:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dfdf80, transno 4295069531 00000004:00080000:0.0:1785220586.227784:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adb800, transno 4295069534 00000004:00080000:0.0:1785220586.227786:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ada300, transno 4295069537 00000004:00080000:0.0:1785220586.227788:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbc700, transno 4295069538 00000004:00080000:0.0:1785220586.227790:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad9f80, transno 4295069539 00000004:00080000:0.0:1785220586.227793:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91581c00, transno 4295069540 00000004:00080000:0.0:1785220586.227796:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbca80, transno 4295069543 00000004:00080000:0.0:1785220586.227799:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e30380, transno 4295069544 00000004:00080000:0.0:1785220586.227801:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbf800, transno 4295069546 00000004:00080000:0.0:1785220586.227803:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e30700, transno 4295069549 00000004:00080000:0.0:1785220586.227805:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bece00, transno 4295069553 00000004:00080000:0.0:1785220586.227808:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e32a00, transno 4295069555 00000004:00080000:0.0:1785220586.227810:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8a2d2a00, transno 4295069556 00000004:00080000:0.0:1785220586.227812:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89dff480, transno 4295069557 00000004:00080000:0.0:1785220586.227814:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97beea00, transno 4295069558 00000004:00080000:0.0:1785220586.227817:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986c380, transno 4295069562 00000004:00080000:0.0:1785220586.227820:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c972680, transno 4295069565 00000004:00080000:0.0:1785220586.227822:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97bed500, transno 4295069566 00000004:00080000:0.0:1785220586.227825:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91582d80, transno 4295069567 00000004:00080000:0.0:1785220586.227828:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97ad8a80, transno 4295069570 00000004:00080000:0.0:1785220586.227831:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adb100, transno 4295069572 00000004:00080000:0.0:1785220586.227833:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cd880, transno 4295069573 00000004:00080000:0.0:1785220586.227835:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958ce680, transno 4295069574 00000004:00080000:0.0:1785220586.227838:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958cd500, transno 4295069575 00000004:00080000:0.0:1785220586.227841:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97adbb80, transno 4295069580 00000004:00080000:0.0:1785220586.227843:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d958ce300, transno 4295069582 00000004:00080000:0.0:1785220586.227846:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98411180, transno 4295069584 00000004:00080000:0.0:1785220586.227849:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98411f80, transno 4295069587 00000004:00080000:0.0:1785220586.227851:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98411c00, transno 4295069588 00000004:00080000:0.0:1785220586.227853:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00dc00, transno 4295069589 00000004:00080000:0.0:1785220586.227855:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991bdc00, transno 4295069590 00000004:00080000:0.0:1785220586.227858:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6f100, transno 4295069593 00000004:00080000:0.0:1785220586.227860:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fe5ca80, transno 4295069594 00000004:00080000:0.0:1785220586.227863:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9362a300, transno 4295069595 00000004:00080000:0.0:1785220586.227865:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6ca80, transno 4295069597 00000004:00080000:0.0:1785220586.227868:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f2a00, transno 4295069603 00000004:00080000:0.0:1785220586.227871:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49c000, transno 4295069604 00000004:00080000:0.0:1785220586.227873:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a9880, transno 4295069605 00000004:00080000:0.0:1785220586.227875:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d991f3800, transno 4295069606 00000004:00080000:0.0:1785220586.227877:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de5c00, transno 4295069607 00000004:00080000:0.0:1785220586.227880:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0a8e00, transno 4295069608 00000004:00080000:0.0:1785220586.227882:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49ca80, transno 4295069610 00000004:00080000:0.0:1785220586.227885:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f49d880, transno 4295069615 00000004:00080000:0.0:1785220586.227887:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92164a80, transno 4295069620 00000004:00080000:0.0:1785220586.227890:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92165180, transno 4295069621 00000004:00080000:0.0:1785220586.227893:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dea00, transno 4295069624 00000004:00080000:0.0:1785220586.227895:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d92165500, transno 4295069626 00000004:00080000:0.0:1785220586.227897:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977de300, transno 4295069627 00000004:00080000:0.0:1785220586.227900:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dd880, transno 4295069629 00000004:00080000:0.0:1785220586.227903:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d901aca80, transno 4295069631 00000004:00080000:0.0:1785220586.227905:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dce00, transno 4295069633 00000004:00080000:0.0:1785220586.227908:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b19d880, transno 4295069635 00000004:00080000:0.0:1785220586.227910:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d977dfb80, transno 4295069638 00000004:00080000:0.0:1785220586.227912:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcaaa00, transno 4295069639 00000004:00080000:0.0:1785220586.227915:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e9f80, transno 4295069642 00000004:00080000:0.0:1785220586.227917:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985ea680, transno 4295069644 00000004:00080000:0.0:1785220586.227919:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bca8700, transno 4295069647 00000004:00080000:0.0:1785220586.227921:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9adb0000, transno 4295069648 00000004:00080000:0.0:1785220586.227924:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9adb1180, transno 4295069649 00000004:00080000:0.0:1785220586.227926:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f54000, transno 4295069652 00000004:00080000:0.0:1785220586.227928:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e8700, transno 4295069653 00000004:00080000:0.0:1785220586.227930:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94f57480, transno 4295069655 00000004:00080000:0.0:1785220586.227933:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480f100, transno 4295069656 00000004:00080000:0.0:1785220586.227935:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7c700, transno 4295069658 00000004:00080000:0.0:1785220586.227937:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480dc00, transno 4295069659 00000004:00080000:0.0:1785220586.227940:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480f800, transno 4295069661 00000004:00080000:0.0:1785220586.227942:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0ec700, transno 4295069666 00000004:00080000:0.0:1785220586.227945:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7d180, transno 4295069667 00000004:00080000:0.0:1785220586.227947:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7dc00, transno 4295069668 00000004:00080000:0.0:1785220586.227949:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7f100, transno 4295069670 00000004:00080000:0.0:1785220586.227951:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb7a300, transno 4295069672 00000004:00080000:0.0:1785220586.227953:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94b99f80, transno 4295069674 00000004:00080000:0.0:1785220586.227956:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0df100, transno 4295069678 00000004:00080000:0.0:1785220586.227959:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94e72300, transno 4295069680 00000004:00080000:0.0:1785220586.227961:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b0dd880, transno 4295069681 00000004:00080000:0.0:1785220586.227964:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908b9180, transno 4295069684 00000004:00080000:0.0:1785220586.227966:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908b8380, transno 4295069685 00000004:00080000:0.0:1785220586.227969:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d908b9c00, transno 4295069687 00000004:00080000:0.0:1785220586.227971:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9856e680, transno 4295069689 00000004:00080000:0.0:1785220586.227973:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90d6c700, transno 4295069692 00000004:00080000:0.0:1785220586.227976:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99376300, transno 4295069693 00000004:00080000:0.0:1785220586.227979:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99376a00, transno 4295069694 00000004:00080000:0.0:1785220586.227981:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99374380, transno 4295069695 00000004:00080000:0.0:1785220586.227983:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a5eed80, transno 4295069696 00000004:00080000:0.0:1785220586.227986:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8f480, transno 4295069697 00000004:00080000:0.0:1785220586.227989:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8d500, transno 4295069699 00000004:00080000:0.0:1785220586.227991:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ad8c000, transno 4295069702 00000004:00080000:0.0:1785220586.227993:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be1880, transno 4295069708 00000004:00080000:0.0:1785220586.227996:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c81180, transno 4295069709 00000004:00080000:0.0:1785220586.227998:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be2a00, transno 4295069710 00000004:00080000:0.0:1785220586.228001:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93742300, transno 4295069711 00000004:00080000:0.0:1785220586.228003:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91505f80, transno 4295069712 00000004:00080000:0.0:1785220586.228006:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91504e00, transno 4295069713 00000004:00080000:0.0:1785220586.228008:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93741180, transno 4295069716 00000004:00080000:0.0:1785220586.228011:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93741500, transno 4295069722 00000004:00080000:0.0:1785220586.228013:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af9c00, transno 4295069723 00000004:00080000:0.0:1785220586.228015:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93742d80, transno 4295069724 00000004:00080000:0.0:1785220586.228018:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676aa00, transno 4295069725 00000004:00080000:0.0:1785220586.228020:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afa300, transno 4295069727 00000004:00080000:0.0:1785220586.228022:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9676b480, transno 4295069728 00000004:00080000:0.0:1785220586.228024:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8f738000, transno 4295069729 00000004:00080000:0.0:1785220586.228026:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cead80, transno 4295069734 00000004:00080000:0.0:1785220586.228029:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984f100, transno 4295069736 00000004:00080000:0.0:1785220586.228032:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984fb80, transno 4295069737 00000004:00080000:0.0:1785220586.228034:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914f480, transno 4295069739 00000004:00080000:0.0:1785220586.228037:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984ce00, transno 4295069740 00000004:00080000:0.0:1785220586.228039:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93571180, transno 4295069744 00000004:00080000:0.0:1785220586.228042:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914ed80, transno 4295069745 00000004:00080000:0.0:1785220586.228045:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cebb80, transno 4295069746 00000004:00080000:0.0:1785220586.228047:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95422680, transno 4295069748 00000004:00080000:0.0:1785220586.228050:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95421c00, transno 4295069749 00000004:00080000:0.0:1785220586.228052:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98afb800, transno 4295069750 00000004:00080000:0.0:1785220586.228054:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91581500, transno 4295069753 00000004:00080000:0.0:1785220586.228056:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91583b80, transno 4295069756 00000004:00080000:0.0:1785220586.228059:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91172680, transno 4295069758 00000004:00080000:0.0:1785220586.228062:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91171500, transno 4295069760 00000004:00080000:0.0:1785220586.228064:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91170700, transno 4295069762 00000004:00080000:0.0:1785220586.228066:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8986c700, transno 4295069767 00000004:00080000:0.0:1785220586.228069:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b45f80, transno 4295069768 00000004:00080000:0.0:1785220586.228071:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9b00c700, transno 4295069769 00000004:00080000:0.0:1785220586.228073:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d638000, transno 4295069771 00000004:00080000:0.0:1785220586.228076:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98b45500, transno 4295069772 00000004:00080000:0.0:1785220586.228078:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95448e00, transno 4295069774 00000004:00080000:0.0:1785220586.228082:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7ce00, transno 4295069780 00000004:00080000:0.0:1785220586.228105:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91172d80, transno 4295069783 00000004:00080000:0.0:1785220586.228109:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0ff800, transno 4295069784 00000004:00080000:0.0:1785220586.228111:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c0fed80, transno 4295069785 00000004:00080000:0.0:1785220586.228114:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98bbca80, transno 4295069786 00000004:00080000:0.0:1785220586.228116:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95448700, transno 4295069788 00000004:00080000:0.0:1785220586.228119:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d89cf9180, transno 4295069789 00000004:00080000:0.0:1785220586.228122:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8d63ad80, transno 4295069790 00000004:00080000:0.0:1785220586.228124:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e86d80, transno 4295069792 00000004:00080000:0.0:1785220586.228127:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af4700, transno 4295069794 00000004:00080000:0.0:1785220586.228130:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfad5500, transno 4295069800 00000004:00080000:0.0:1785220586.228133:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979a7b80, transno 4295069801 00000004:00080000:0.0:1785220586.228136:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e0380, transno 4295069802 00000004:00080000:0.0:1785220586.228139:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e0aa300, transno 4295069804 00000004:00080000:0.0:1785220586.228141:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90e6e680, transno 4295069805 00000004:00080000:0.0:1785220586.228144:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e8e00, transno 4295069808 00000004:00080000:0.0:1785220586.228147:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d99026a00, transno 4295069809 00000004:00080000:0.0:1785220586.228149:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c571880, transno 4295069810 00000004:00080000:0.0:1785220586.228152:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9adb0700, transno 4295069815 00000004:00080000:0.0:1785220586.228154:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8914dc00, transno 4295069817 00000004:00080000:0.0:1785220586.228157:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9adb1500, transno 4295069820 00000004:00080000:0.0:1785220586.228160:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa66a00, transno 4295069821 00000004:00080000:0.0:1785220586.228163:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa65180, transno 4295069822 00000004:00080000:0.0:1785220586.228165:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8fa64380, transno 4295069823 00000004:00080000:0.0:1785220586.228168:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb7aa00, transno 4295069824 00000004:00080000:0.0:1785220586.228171:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98af8700, transno 4295069827 00000004:00080000:0.0:1785220586.228174:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97589880, transno 4295069832 00000004:00080000:0.0:1785220586.228177:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87d44000, transno 4295069834 00000004:00080000:0.0:1785220586.228179:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480d180, transno 4295069835 00000004:00080000:0.0:1785220586.228182:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9758aa00, transno 4295069836 00000004:00080000:0.0:1785220586.228185:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9553ad80, transno 4295069839 00000004:00080000:0.0:1785220586.228188:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af4e00, transno 4295069840 00000004:00080000:0.0:1785220586.228191:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97588e00, transno 4295069846 00000004:00080000:0.0:1785220586.228194:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faa5880, transno 4295069847 00000004:00080000:0.0:1785220586.228197:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e9880, transno 4295069849 00000004:00080000:0.0:1785220586.228200:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9480c380, transno 4295069852 00000004:00080000:0.0:1785220586.228202:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97589f80, transno 4295069853 00000004:00080000:0.0:1785220586.228205:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d96768a80, transno 4295069854 00000004:00080000:0.0:1785220586.228207:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d90c80000, transno 4295069855 00000004:00080000:0.0:1785220586.228210:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e770700, transno 4295069856 00000004:00080000:0.0:1785220586.228213:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040d500, transno 4295069862 00000004:00080000:0.0:1785220586.228215:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d979bad80, transno 4295069864 00000004:00080000:0.0:1785220586.228218:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9a6e0000, transno 4295069865 00000004:00080000:0.0:1785220586.228220:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040ed80, transno 4295069866 00000004:00080000:0.0:1785220586.228223:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94d7c000, transno 4295069867 00000004:00080000:0.0:1785220586.228226:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040f800, transno 4295069869 00000004:00080000:0.0:1785220586.228228:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040c000, transno 4295069871 00000004:00080000:0.0:1785220586.228231:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040ea00, transno 4295069872 00000004:00080000:0.0:1785220586.228233:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040fb80, transno 4295069877 00000004:00080000:0.0:1785220586.228236:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040f100, transno 4295069879 00000004:00080000:0.0:1785220586.228238:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040d880, transno 4295069880 00000004:00080000:0.0:1785220586.228241:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc1180, transno 4295069881 00000004:00080000:0.0:1785220586.228244:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8faab100, transno 4295069883 00000004:00080000:0.0:1785220586.228246:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040dc00, transno 4295069886 00000004:00080000:0.0:1785220586.228248:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040c700, transno 4295069887 00000004:00080000:0.0:1785220586.228251:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95470a80, transno 4295069888 00000004:00080000:0.0:1785220586.228253:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8984ed80, transno 4295069889 00000004:00080000:0.0:1785220586.228256:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfb35500, transno 4295069890 00000004:00080000:0.0:1785220586.228258:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98581c00, transno 4295069892 00000004:00080000:0.0:1785220586.228261:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa5bb80, transno 4295069894 00000004:00080000:0.0:1785220586.228264:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95470700, transno 4295069895 00000004:00080000:0.0:1785220586.228266:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cfe9c00, transno 4295069899 00000004:00080000:0.0:1785220586.228269:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d93649180, transno 4295069902 00000004:00080000:0.0:1785220586.228271:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8bcab100, transno 4295069904 00000004:00080000:0.0:1785220586.228274:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8cb85880, transno 4295069905 00000004:00080000:0.0:1785220586.228276:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbf480, transno 4295069908 00000004:00080000:0.0:1785220586.231209:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d91582680, transno 4295069909 00000004:00080000:0.0:1785220586.231220:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95473800, transno 4295069910 00000004:00080000:0.0:1785220586.231223:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86310e00, transno 4295069911 00000004:00080000:0.0:1785220586.231226:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98583100, transno 4295069912 00000004:00080000:0.0:1785220586.231229:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d95470380, transno 4295069914 00000004:00080000:0.0:1785220586.231231:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98583b80, transno 4295069921 00000004:00080000:0.0:1785220586.231234:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d98581180, transno 4295069922 00000004:00080000:0.0:1785220586.231237:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e771500, transno 4295069925 00000004:00080000:0.0:1785220586.231240:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d0000, transno 4295069927 00000004:00080000:0.0:1785220586.231242:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6bb80, transno 4295069928 00000004:00080000:0.0:1785220586.231245:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae68700, transno 4295069931 00000004:00080000:0.0:1785220586.231248:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d987d3480, transno 4295069933 00000004:00080000:0.0:1785220586.231250:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae68380, transno 4295069936 00000004:00080000:0.0:1785220586.231252:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae69180, transno 4295069938 00000004:00080000:0.0:1785220586.231255:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae68a80, transno 4295069940 00000004:00080000:0.0:1785220586.231258:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6ad80, transno 4295069941 00000004:00080000:0.0:1785220586.231260:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae68000, transno 4295069943 00000004:00080000:0.0:1785220586.231262:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9ae6b800, transno 4295069949 00000004:00080000:0.0:1785220586.231265:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97be1c00, transno 4295069950 00000004:00080000:0.0:1785220586.231268:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9758b800, transno 4295069951 00000004:00080000:0.0:1785220586.231271:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040ce00, transno 4295069953 00000004:00080000:0.0:1785220586.231273:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94bc0380, transno 4295069957 00000004:00080000:0.0:1785220586.231411:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e86313b80, transno 4295069958 00000004:00080000:0.0:1785220586.231414:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d97af7b80, transno 4295069959 00000004:00080000:0.0:1785220586.231417:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a1f80, transno 4295069960 00000004:00080000:0.0:1785220586.231420:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d985e8380, transno 4295069963 00000004:00080000:0.0:1785220586.231422:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d934a2680, transno 4295069966 00000004:00080000:0.0:1785220586.231425:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8c92b100, transno 4295069968 00000004:00080000:0.0:1785220586.231427:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d94cbed80, transno 4295069969 00000004:00080000:0.0:1785220586.231430:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9758ad80, transno 4295069973 00000004:00080000:0.0:1785220586.231432:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d9040d180, transno 4295069974 00000004:00080000:0.0:1785220586.231435:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa58a80, transno 4295069975 00000004:00080000:0.0:1785220586.231437:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960ebfa58000, transno 4295069976 00000004:00080000:0.0:1785220586.231439:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d8e770380, transno 4295069982 00000004:00080000:0.0:1785220586.231442:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960d87de4380, transno 4295069983 00000100:00100000:0.0:1785220586.231466:0:3652:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d93608380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1871938510816512:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1785220586.232609:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1871938510817792:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.232657:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d984e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4add7b40-ab9d-45be-a6c9-eb07929d24cd:3655:1871938510816640:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.232675:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1871938510817152:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.232700:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1871938510817664:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.232713:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d984e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1871938510816768:0@lo:400:kworker.0 00000100:00100000:2.0:1785220586.232725:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1871938510817280:0@lo:400:kworker.0 00000100:00100000:3.0:1785220586.235973:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d8e5b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510817024:0@lo:400:kworker.0 00000004:00080000:0.0:1785220586.239212:0:8746:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0001: 0 changes, 0 in progress, 0 in flight, 596 done 00000004:00080000:1.0:1785220586.247579:0:9936:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0001: 0 changes, 0 in progress, 0 in flight, 496 done 00000100:00100000:0.0:1785220586.729626:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1785220586.729653:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9360aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510818176:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1785220586.729735:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220586.729913:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510818176 00000100:00100000:1.0:1785220586.729922:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510818176 00000100:00100000:1.0:1785220586.729976:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:1.0:1785220586.729985:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+66:3655:x1871938510818176:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1785220586.730018:0:8444:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 286720 granted 582591936 pending 57344 free 3891458048 avail 3659935744 00000100:00100000:1.0:1785220586.730075:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a707480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+66:3655:x1871938510818176:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (340us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220586.730082:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:0.0:1785220586.730183:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9360aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1871938510818176:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1785220586.730438:0:3655:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d9679b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1871938510818304:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1785220586.730470:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1785220586.730515:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510818304 00000100:00100000:1.0:1785220586.730519:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510818304 00000100:00100000:1.0:1785220586.730553:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12577 00000100:00100000:1.0:1785220586.730560:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3655:x1871938510818304:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1785220586.730585:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1439232000 avail 1292992512 00000100:00100000:1.0:1785220586.730618:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3655:x1871938510818304:12345-0@lo:41:osp-pre-0-1.0 Request processed in 57us (146us total) trans 0 rc 0/0 00000100:00100000:1.0:1785220586.730624:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12577 00000100:00100000:0.0:1785220586.730724:0:3655:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d9679b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1871938510818304:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1785220586.798270:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1785220587.621612:0:14346:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: racer test_2: @@@@@@ FAIL: test_2 failed with 1 00000100:00100000:2.0:1785220587.755346:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1785220587.755370:0:3653:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff960d96798000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510818432:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1785220587.755440:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1785220587.758165:0:13845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938510818432 00000100:00100000:0.0:1785220587.758178:0:13845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938510818432 00000100:00100000:0.0:1785220587.758287:0:13845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12578 00000100:00100000:0.0:1785220587.758296:0:13845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9679bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3653:x1871938510818432:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1785220587.758331:0:13845:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4243456 pending 0 free 1440956416 avail 1294716928 00000100:00100000:0.0:1785220587.758398:0:13845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9679bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3653:x1871938510818432:12345-0@lo:41:osp-pre-1-0.0 Request processed in 104us (2958us total) trans 0 rc 0/0 00000100:00100000:0.0:1785220587.758405:0:13845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12578 00000100:00100000:3.0:1785220587.759162:0:3653:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff960d96798000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1871938510818432:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1785220587.765984:0:8435:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4295070834 is committed 00000001:00080000:1.0:1785220587.768903:0:9520:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4295070299 is committed 00000001:00080000:1.0:1785220587.768924:0:9520:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4295070297 is committed 00010000:00010000:1.0:1785220587.817331:0:10345:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1785220588.614780:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615113:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615148:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615179:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615199:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615218:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615251:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615270:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615288:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.615307:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220588.615413:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520029952 00000100:00100000:3.0:1785220588.615423:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520029952 00000100:00100000:3.0:1785220588.615474:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 610 00000100:00100000:3.0:1785220588.615483:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960eac529050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e3216512-9662-458b-a984-8fbdc3fc5dc2+13:2406:x1871938520029952:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.615552:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960eac529050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e3216512-9662-458b-a984-8fbdc3fc5dc2+13:2406:x1871938520029952:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 70us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.615559:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 610 00000100:00100000:3.0:1785220588.615698:0:13806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030208 00000100:00100000:3.0:1785220588.615703:0:13806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030208 00000100:00100000:3.0:1785220588.615738:0:13806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030080 00000100:00100000:3.0:1785220588.615741:0:13806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030080 00000100:00100000:3.0:1785220588.615758:0:13806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030592 00000100:00100000:3.0:1785220588.615760:0:13806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030592 00000100:00100000:3.0:1785220588.615791:0:13806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1360 00000100:00100000:3.0:1785220588.615798:0:13806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+479:2406:x1871938520030208:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.615846:0:13806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+479:2406:x1871938520030208:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 49us (697us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.615853:0:13806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1360 00000100:00100000:3.0:1785220588.615867:0:13806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1361 00000100:00100000:3.0:1785220588.615873:0:13806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:2406:x1871938520030080:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.615912:0:13806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:2406:x1871938520030080:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 40us (693us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.615918:0:13806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1361 00000100:00100000:3.0:1785220588.615931:0:13806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1362 00000100:00100000:3.0:1785220588.615937:0:13806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+6:2406:x1871938520030592:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.615960:0:13806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+6:2406:x1871938520030592:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 23us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.615969:0:13806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1362 00000100:00100000:3.0:1785220588.616009:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030464 00000100:00100000:3.0:1785220588.616013:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030464 00000100:00100000:3.0:1785220588.616039:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030720 00000100:00100000:3.0:1785220588.616042:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030720 00000100:00100000:3.0:1785220588.616057:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030336 00000100:00100000:3.0:1785220588.616059:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030336 00000100:00100000:3.0:1785220588.616074:0:14202:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030848 00000100:00100000:3.0:1785220588.616076:0:14202:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030848 00000100:00100000:3.0:1785220588.616120:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 435 00000100:00100000:3.0:1785220588.616127:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:5644b1e7-55e7-4421-a7e4-714e45fd656f+697:2406:x1871938520030464:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.616155:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a705c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:5644b1e7-55e7-4421-a7e4-714e45fd656f+697:2406:x1871938520030464:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 29us (975us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.616161:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 435 00000100:00100000:3.0:1785220588.616173:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 436 00000100:00100000:3.0:1785220588.616179:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d97b39c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2406:x1871938520030720:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.616202:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d97b39c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+20:2406:x1871938520030720:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 23us (1002us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.616207:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 436 00000100:00100000:3.0:1785220588.616218:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 437 00000100:00100000:3.0:1785220588.616300:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:2406:x1871938520030336:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.616327:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:5644b1e7-55e7-4421-a7e4-714e45fd656f+691:2406:x1871938520030336:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 103us (1076us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.616333:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 437 00000100:00100000:3.0:1785220588.616345:0:14202:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 438 00000100:00100000:3.0:1785220588.616352:0:14202:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+10:2406:x1871938520030848:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:3.0:1785220588.616373:0:14202:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_058:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+10:2406:x1871938520030848:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 22us (1085us total) trans 0 rc 0/0 00000100:00100000:3.0:1785220588.616378:0:14202:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 438 00000100:00080000:3.0:1785220588.616507:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1785220588.616512:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1785220588.616515:0:6522:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1785220588.622372:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520029824 00000100:00100000:2.0:1785220588.622389:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520029824 00000100:00100000:2.0:1785220588.622453:0:13815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030976 00000100:00100000:2.0:1785220588.622457:0:13815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030976 00000100:00100000:2.0:1785220588.622489:0:13815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 12579 00000100:00100000:2.0:1785220588.622499:0:13815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2404:x1871938520029824:12345-192.168.201.51@tcp:63:ll_sa.0 00010000:00010000:2.0:1785220588.622532:0:13815:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1785220588.622552:0:13815:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92e00/0xf62499ab927a60a2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdff expref: -99 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1785220588.622723:0:13914:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 12580 00000100:00100000:3.0:1785220588.622735:0:13914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d8a2d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:2407:x1871938520030976:12345-192.168.201.51@tcp:63:ll_sa.0 00010000:00010000:3.0:1785220588.622753:0:13914:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1785220588.622763:0:13914:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68a00/0xf62499ab927a60b0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fe06 expref: -99 pid: 13914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622821:0:13914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622835:0:13914:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622849:0:13914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622863:0:13914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622871:0:13914:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622877:0:13914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622923:0:13914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622931:0:13914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622937:0:13914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622943:0:13914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea69400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220588.622947:0:13914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622954:0:13914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40218400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622965:0:13914:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:3.0:1785220588.622968:0:13914:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220588.622973:0:13914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622979:0:13914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea69400/0xf62499ab927a60b7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622992:0:13914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.622998:0:13914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623005:0:13914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623010:0:13914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d8ea68800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:3.0:1785220588.623013:0:13914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623018:0:13914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623025:0:13914:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0xd964d85e (ffff960d8e268700) 00010000:00010000:3.0:1785220588.623027:0:13914:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1785220588.623030:0:13914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623036:0:13914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68800/0xf62499ab927a60be lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13914 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623076:0:13914:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68a00/0xf62499ab927a60b0 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fe06 expref: 589 pid: 13914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623131:0:13914:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:3.0:1785220588.623134:0:13914:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1785220588.623138:0:13914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d8ea68a00/0xf62499ab927a60b0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fe06 expref: 589 pid: 13914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1785220588.623149:0:13914:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d8ea68a00, rc -2) 00000100:00100000:3.0:1785220588.623369:0:13914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d8a2d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_005:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+588:2407:x1871938520030976:12345-192.168.201.51@tcp:63:ll_sa.0 Request processed in 492us (7914us total) trans 0 rc -2/-2 00000100:00100000:3.0:1785220588.623381:0:13914:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 12580 00010000:00010000:2.0:1785220588.624334:0:13815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624355:0:13815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624367:0:13815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624389:0:13815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624395:0:13815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624401:0:13815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624439:0:13815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624444:0:13815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624450:0:13815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624456:0:13815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b92800 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220588.624460:0:13815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624466:0:13815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624475:0:13815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:2.0:1785220588.624477:0:13815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220588.624481:0:13815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624487:0:13815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92800/0xf62499ab927a60a9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624496:0:13815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624501:0:13815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624506:0:13815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624511:0:13815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d90b93600 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:2.0:1785220588.624513:0:13815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624519:0:13815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624525:0:13815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x85306dec (ffff960d94863100) 00010000:00010000:2.0:1785220588.624527:0:13815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1785220588.624530:0:13815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624536:0:13815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b93600/0xf62499ab927a60c5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624579:0:13815:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92e00/0xf62499ab927a60a2 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdff expref: 587 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624594:0:13815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:2.0:1785220588.624596:0:13815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1785220588.624599:0:13815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d90b92e00/0xf62499ab927a60a2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdff expref: 587 pid: 13815 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1785220588.624608:0:13815:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d90b92e00, rc -2) 00000100:00100000:2.0:1785220588.624680:0:13815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+586:2404:x1871938520029824:12345-192.168.201.51@tcp:63:ll_sa.0 Request processed in 2182us (9910us total) trans 0 rc -2/-2 00000100:00100000:2.0:1785220588.624689:0:13815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 12579 00000100:00100000:3.0:1785220588.630000:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1785220588.631381:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:0.0:1785220588.631526:0:13845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520030976 00000100:00100000:0.0:1785220588.631535:0:13845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520030976 00000100:00100000:0.0:1785220588.631594:0:13845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1871938520029824 00000100:00100000:0.0:1785220588.631598:0:13845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1871938520029824 00000100:00100000:0.0:1785220588.631623:0:13845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 12581 00000100:00100000:0.0:1785220588.631634:0:13845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2407:x1871938520030976:12345-192.168.201.51@tcp:63:ll_sa.0 00000020:00080000:0.0:1785220588.631690:0:13845:0:(tgt_handler.c:3159:tgt_check_resent()) @@@ no reply for RESENT req req@ffff960d9a5eea00 x1871938520030976/t0(0) o63->45383443-d1a2-4f43-8eb5-d11e3b9f90c7@192.168.201.51@tcp:581/0 lens 664/4320 e 0 to 0 dl 1785220731 ref 1 fl Interpret:/202/0 rc 0/0 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1785220588.631704:0:13845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1785220588.631981:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 12582 00000100:00100000:1.0:1785220588.631991:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff960d9a5eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2404:x1871938520029824:12345-192.168.201.51@tcp:63:ll_sa.0 00010000:00010000:0.0:1785220588.632204:0:13845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a60cc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fe06 expref: -99 pid: 13845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1785220588.632368:0:13845:0:(mdt_handler.c:4960:mdt_intent_fixup_resent()) @@@ no existing lock with rhandle 0x384cc2c1b7fe06 req@ffff960d9a5eea00 x1871938520030976/t0(0) o63->45383443-d1a2-4f43-8eb5-d11e3b9f90c7@192.168.201.51@tcp:581/0 lens 664/53136 e 0 to 0 dl 1785220731 ref 1 fl Interpret:/200/0 rc 0/0 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1785220588.632392:0:13845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632408:0:13845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632420:0:13845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632434:0:13845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632441:0:13845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632447:0:13845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632487:0:13845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632494:0:13845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632500:0:13845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632506:0:13845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be3a00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220588.632510:0:13845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632517:0:13845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632530:0:13845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:0.0:1785220588.632533:0:13845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220588.632537:0:13845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632544:0:13845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be3a00/0xf62499ab927a60d3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632556:0:13845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632563:0:13845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632568:0:13845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632572:0:13845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d94be2e00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:0.0:1785220588.632575:0:13845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632584:0:13845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632592:0:13845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0xd964d85e (ffff960d8adc7600) 00010000:00010000:0.0:1785220588.632594:0:13845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1785220588.632597:0:13845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632602:0:13845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2e00/0xf62499ab927a60da lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0xd964d85e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632654:0:13845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a60cc lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fe06 expref: 588 pid: 13845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632664:0:13845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:0.0:1785220588.632666:0:13845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1785220588.632670:0:13845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d94be2600/0xf62499ab927a60cc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fe06 expref: 588 pid: 13845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1785220588.632680:0:13845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d94be2600, rc -2) 00000020:00080000:1.0:1785220588.633427:0:6531:0:(tgt_handler.c:3159:tgt_check_resent()) @@@ no reply for RESENT req req@ffff960d9a5eca80 x1871938520029824/t0(0) o63->45383443-d1a2-4f43-8eb5-d11e3b9f90c7@192.168.201.51@tcp:581/0 lens 664/4320 e 0 to 0 dl 1785220731 ref 1 fl Interpret:/202/0 rc 0/0 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1785220588.633442:0:6531:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1785220588.633459:0:6531:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a60e1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x384cc2c1b7fdff expref: -99 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1785220588.633533:0:6531:0:(mdt_handler.c:4960:mdt_intent_fixup_resent()) @@@ no existing lock with rhandle 0x384cc2c1b7fdff req@ffff960d9a5eca80 x1871938520029824/t0(0) o63->45383443-d1a2-4f43-8eb5-d11e3b9f90c7@192.168.201.51@tcp:581/0 lens 664/53136 e 0 to 0 dl 1785220731 ref 1 fl Interpret:/200/0 rc 0/0 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1785220588.633555:0:6531:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633567:0:6531:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633577:0:6531:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633592:0:6531:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633599:0:6531:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633607:0:6531:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633635:0:6531:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633656:0:6531:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633664:0:6531:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633669:0:6531:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496c00 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220588.633674:0:6531:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633681:0:6531:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633691:0:6531:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:1.0:1785220588.633693:0:6531:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220588.633697:0:6531:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633703:0:6531:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496c00/0xf62499ab927a60e8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633715:0:6531:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633722:0:6531:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633740:0:6531:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633745:0:6531:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960d98496400 already unused, calling callback (ffffffffc1a684f0) 00010000:00010000:1.0:1785220588.633748:0:6531:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633753:0:6531:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633760:0:6531:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x85306dec (ffff960d8a305f00) 00010000:00010000:1.0:1785220588.633762:0:6531:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1785220588.633765:0:6531:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633770:0:6531:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496400/0xf62499ab927a60ef lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2:0x0].0x85306dec bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633801:0:6531:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a60e1 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdff expref: 588 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633810:0:6531:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960e8aaf6c00) 00010000:00010000:1.0:1785220588.633813:0:6531:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1785220588.633816:0:6531:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960d98496200/0xf62499ab927a60e1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.51@tcp remote: 0x384cc2c1b7fdff expref: 588 pid: 6531 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1785220588.633825:0:6531:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960d98496200, rc -2) 00000100:00100000:0.0:1785220588.634041:0:13845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+587:2407:x1871938520030976:12345-192.168.201.51@tcp:63:ll_sa.0 Request processed in 2404us (4046us total) trans 0 rc -2/-2 00000100:00100000:0.0:1785220588.634051:0:13845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 12581 00000100:00100000:1.0:1785220588.634207:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff960d9a5eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:45383443-d1a2-4f43-8eb5-d11e3b9f90c7+586:2404:x1871938520029824:12345-192.168.201.51@tcp:63:ll_sa.0 Request processed in 2218us (2831us total) trans 0 rc -2/-2 00000100:00100000:1.0:1785220588.634215:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 12582 00010000:00010000:2.0:1785220588.842732:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 60111 lines, 60111 kept, 0 dropped, 0 bad.